JP2015097074A - Information processing apparatus, control method of the same, and program, and projection system, control method of the same, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an apparatus that controls an imaging sound of an imaging device in obtaining a trace.SOLUTION: In a projection system 100 that is communicably connected with a camera 102, infrared sensor 103, and projector 104, an information processing apparatus 101 determines whether or not an object such as a finger of a user is an object that leaves a trace. When determined that the object is the object that leaves a trace such as a finger of a user, the information processing apparatus 101 reduces an imaging sound of the camera 102, and when determined that the object is not the object that leaves a trace such as a finger of a user, outputs the imaging sound of the camera 102.

Description

  The present invention relates to an information processing apparatus capable of leaving a trail that can identify a user who has performed an operation on projected image data, a control method and program thereof, and a projection system, and a control method and program thereof.

  Conventionally, when a financial product is sold or explained, a person who explains the financial product (hereinafter referred to as a teller) has given the customer a paper medium such as a catalog of the financial product. In the paper catalog, Teller could leave an explanatory record for each customer, but could not leave an objective trail record explaining what part and when.

  Therefore, if there is a portion of the paper medium that the teller has forgotten to explain, the customer cannot recognize that portion. Even if a customer signs and seals, it does not mean that they have received explanations for all places where accountability is required, but can only assure that they have received a single explanation from the teller, and the substance has become a substantial body. It was.

  Under such circumstances, since paper media cannot leave an objective history, when customers insist that they are not explaining financial products, they fall into the watering theory of “explained, not explained” End up. In order to avoid this, there was a need for a way for financial institutions to ensure accountability as required by law.

  Therefore, as a method for proving that the teller has explained, the following patent document 1 describes the trail explained to the customer by recording the log at the window terminal device used when the teller explains. The mechanism to do is disclosed.

JP 2009-122796 A

  Also, as a way to prove that Teller has explained, it is possible to save the trail by saving the state of explanation as a movie with a camera etc. or saving it as a still image at a key point Has been done. In addition, it is also done to leave a trail of consenting act such as a customer pressing a sign or approval button.

  However, the image pickup sound generated for taking a trail is not so much mentally pleasant for the customer, and clearly gives a sense that the trail is being taken. Such a sense negatively affected the trust relationship with Teller.

  Therefore, a mechanism for eliminating the imaging sound of the imaging device or reducing the imaging sound is conceivable, but in the projection system 100 as shown in FIG. 4, the camera 102 (imaging device) plays the role of leaving the trail described above, The imaging apparatus also has a role of capturing a paper document placed on the projection surface 106 and capturing it as image data in the information processing apparatus 101. For this reason, when a paper document is captured by the imaging device, there is a problem in that it is not possible to know whether or not the paper document has been shot unless an imaging sound is heard. That is, the image pickup sound of the image pickup apparatus cannot be simply turned off or the image pickup sound cannot be reduced.

  In addition, when there is only one kind of image pickup sound, the image pickup sound is not switched depending on the image pickup scene, so that the user such as a teller or a customer cannot understand the meaning of the image pickup. Each time an image is taken, the customer loses concentration on the teller's explanation by inferring the meaning of the image from his actions and the actions of the other party.

  For example, in the trail, there are an explanation trail that guarantees that the teller explained to the customer, and an agreement trail that guarantees that the consent has been given, and there is one imaging sound for each trail preservation In this case, there is a problem that the user cannot understand the reason why the image pickup sound is generated.

  Furthermore, if the captured sound does not change not only when the trail image is stored but also when a paper document is captured by the imaging device, the user cannot understand the meaning of the captured image only by the captured sound, and must think about the meaning one by one. Don't be.

  Therefore, a first object of the present invention is to control imaging sound that is output in accordance with imaging by an imaging device when taking a trail.

  Furthermore, the second object of the present invention is to switch the image pickup sound output according to the image pickup of the image pickup device according to the scene where the object on the projection plane is picked up.

  In order to achieve the first object, an information processing apparatus of the present invention images a projection device that projects an image and an object that is a real object that exists on a projection plane projected by the projection device. An information processing apparatus that is communicably connected to an imaging apparatus, the object specifying unit specifying the type of the object, and depending on the imaging of the imaging apparatus according to the type of the object specified by the object specifying unit Imaging sound control means for controlling the imaging sound output in this manner.

  In order to achieve the second object, an information processing apparatus according to the present invention is an information processing apparatus that is communicably connected to a projection apparatus that projects electronic data and an imaging apparatus that captures an image according to an imaging instruction. When the user's action is detected by the storage means for storing the picked-up sound for each process executed according to the user's action, the action detecting means for detecting the user action, and the action detecting means, An imaging instruction transmission unit that transmits an imaging instruction to the imaging device, and an imaging sound output unit that outputs an imaging sound in response to the imaging instruction transmitted by the imaging instruction transmission unit, the imaging sound output unit The image pickup sound corresponding to the process executed in accordance with the operation detected by the operation detection unit is acquired from the storage unit and output.

  According to the present invention, there is a first effect that enables the imaging sound of the imaging apparatus to be controlled when taking a trail.

  According to the present invention, it is possible to switch the image pickup sound output according to the image pickup of the image pickup apparatus according to the scene where the object on the projection plane is picked up.

1 is a configuration diagram showing an example of a projection system 100. FIG. FIG. 2 is a configuration diagram illustrating an example of a hardware configuration of a projection system 100. 2 is a configuration diagram illustrating an example of a module configuration of an information processing apparatus 101. FIG. 1 is a configuration diagram showing an example of a projection system 100. FIG. It is a block diagram which shows an example of the flow of a series of processes in embodiment of this invention. It is an example of a screen which shows an example of the screen which receives ID input displayed after starting a trail recording application. 3 is a configuration diagram illustrating an example of a configuration of a teller ID table 700 stored in an information processing apparatus 101. FIG. 3 is a configuration diagram illustrating an example of a configuration of a customer ID table 800 stored in an information processing apparatus 101. FIG. It is a flowchart which shows the detail of the projection process shown in FIG. It is a schematic diagram which shows an example at the time of performing a projection process in FIG. It is a flowchart which shows the detail of the touch determination process shown in FIG. 3 is a configuration diagram illustrating an example of a configuration of an imaging region information table 1200 stored in an information processing apparatus 101. FIG. FIG. 10 is a schematic diagram illustrating an example of a case where a teller 1001 touches an outline description portion 1004 with a projection image facing a customer 1002; It is a schematic diagram which shows an example in case the customer 1002 is touching the summary description part 1004 with the projection image facing the customer 1002. FIG. 10 is a schematic diagram illustrating an example of a case where the teller 1001 is touching an outline description portion 1004 with a projection image facing the teller 1001; FIG. 10 is a schematic diagram showing an example of a case where a customer 1002 signs a projected image in the signature entry field 1007 in the direction of the customer 1002. FIG. 10 is a schematic diagram showing an example of a case where the teller 1001 signs a projected image in the signature entry field 1007 in the direction of the teller 1001. FIG. 10 is a schematic diagram illustrating an example of a case where a teller 1001 signs a projected image in a signature entry field 1007 for a customer 1002; 6 is a configuration diagram illustrating an example of a configuration of a trail storage table 1900 stored in an information processing apparatus 101. FIG. 6 is a flowchart showing details of a paper document capturing process shown in FIG. 5. It is a schematic diagram which shows an example in case a customer's resident's card 2102 is imaged and taken in as image data. It is a block diagram which shows an example of a structure of the condition table 2200 which shows the conditions which perform an imaging instruction | indication with respect to the camera. It is a block diagram which shows an example of the structure of the projection table 2300 which stores the projection document in projection and the rotation flag of the said projection document. It is a block diagram which shows an example of the flow of a series of processes of the projection process in a 2nd Example. It is a block diagram which shows an example of a series of flows of the touch determination process in a 2nd Example. It is a block diagram which shows an example of the hardware constitutions of the projection system 100 in a 3rd Example. It is a block diagram which shows an example of a series of flows of the touch determination process in a 3rd Example. It is a block diagram which shows an example of a series of flows of the paper document taking-in process in a 3rd Example. It is the imaging sound data table 2900 which stores the imaging sound for every scene for switching an imaging sound with the imaging scene in a 3rd Example.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a diagram showing an example of a system configuration of a projection system 100 according to the present invention. The projection system 100 of the present invention includes an information processing apparatus 101, a camera 102, an infrared sensor 103, a projector 104, and a printer 107. The information processing apparatus 101, camera 102, infrared sensor 103, projector 104, and printer 107 are connected to a USB. (Universal Serial Bus), a VGA terminal, and the like are connected so as to be capable of data communication with each other. FIG. 4 shows the projection system 100 shown in FIG. 1 in three dimensions. In this manner, a device including the camera 102, the infrared sensor 103, and the projector 104 is installed on a desk. In the present embodiment, description is made on the premise of wired connection, but wireless connection may be used. The configuration of the various terminals in FIG. 1 is an example, and there are various configuration examples depending on applications and purposes.

  The projection system 100 is a system that allows a user to give a predetermined instruction to an image projected on the projection surface 106. For example, the user touches a button projected on the projection surface 106 to execute predetermined processing, or a paper document is placed on the projection surface 106 to generate image data of the paper document, and the generated image data Can be projected onto the projection plane 106. By using such a system, a plurality of users can see one image projected on the projection surface 106, so that information can be shared smoothly and a plurality of images can be shared with respect to the projected image. Can be directly operated. The same effect can be obtained with a tablet terminal having a touch panel. However, since the display is small, there is a problem that it is difficult for a plurality of users to browse simultaneously. Therefore, this problem is solved by projecting an image on the projection surface 106 such as a desk so that the projected image can be directly operated. In the following, devices constituting this system will be described.

  The information processing apparatus 101 is an apparatus such as a personal computer or a server. As described above, the information processing apparatus 101 is connected to the camera 102, the infrared sensor 103, and the projector 104, and can exchange information with these devices. Since the information processing apparatus 101 controls the projection system 100, the projector 104 is instructed to project various data stored in the information processing apparatus 101, and the camera 102 is instructed to shoot. Thus, image data can be obtained and information obtained from the infrared sensor 103 can be analyzed. For example, the information processing apparatus 101 issues an imaging instruction to the camera 102 based on information obtained from the infrared sensor 103, stores image data obtained from the camera 102 in the RAM 203 of the information processing apparatus 101, and an image stored in the RAM 203. Data is transmitted to the projector 104. Further, the information processing apparatus 101 is a projector based on information on an actual object (hereinafter referred to as an object) such as a user's hand or a paper document obtained from the infrared sensor 103, information on the operation of the object, or the like. Processing such as enlarging, reducing, or hiding the image projected from 104 to the projection surface 106 is performed. These processes are operated by a trail recording application installed in the information processing apparatus 101. The trail recording application is an application for transmitting / receiving information to / from various devices described above and receiving instructions from the user. By operating this trail recording application, a series of processing of this embodiment is realized.

  The camera 102 (imaging device) is a digital camera that can capture a moving image or a still image, and receives an imaging instruction from the information processing device 101 to capture an imaging range of the camera 102. Image data obtained by imaging is transmitted to the information processing apparatus 101.

  The infrared sensor 103 is, for example, an infrared sensor capable of photographing an infrared light region, acquires information on which infrared light is reflected from the projection surface 106, and transmits an infrared electrical signal (hereinafter referred to as infrared information) to the information processing apparatus 101. To do. The information processing apparatus 101 identifies the shape, movement, position, etc. of the object based on the received information. In this embodiment, an infrared sensor is used. However, the type of sensor is not particularly limited as long as the shape, movement, position, etc. of the object can be specified.

  The projector 104 (projection device) is a device that projects various data sent from the information processing device 101. The projector 104 is installed toward the reflecting mirror 105. When receiving a projection instruction from the information processing apparatus 101, the projector 104 projects toward the reflecting mirror 105, and projects onto the projection surface 106 through the reflection of the reflecting mirror 105. .

  The printer 107 is a device that performs printing based on a print job sent from the information processing apparatus 101.

  FIG. 2 is a diagram illustrating a hardware configuration of various terminals according to the embodiment of the present invention.

  The CPU 201 comprehensively controls each device and controller connected to the system bus 204.

  Further, the ROM 202 or the external memory 211 is necessary to realize a BIOS (Basic Input / Output System) that is a control program of the CPU 201, an operating system program (hereinafter referred to as OS), and a function executed by each server or each PC. Various programs to be described later are stored. The RAM 203 functions as a main memory, work area, and the like for the CPU 201.

  The CPU 201 implements various operations by loading a program necessary for execution of processing into the RAM 203 and executing the program.

  An input controller (input C) 205 controls input from an input device 209 such as a keyboard or a pointing device such as a mouse (not shown). The camera 102 and the infrared sensor 103 are also connected to control input of image data and infrared information transmitted from these devices.

  A video controller (VC) 206 controls display on a display device such as the display 210. The type of the display device is assumed to be a CRT or a liquid crystal display, but is not limited thereto. Further, the projector 104 is also connected to control the projection on the projector 104.

  The memory controller (MC) 207 is an adapter to a hard disk (HD), flexible disk (FD) or PCMCIA card slot for storing boot programs, browser software, various applications, font data, user files, editing files, various data, and the like. Controls access to an external memory 211 such as a card-type memory connected via the.

  A communication I / F controller (communication I / FC) 208 is connected to and communicates with an external device via a network, and executes communication control processing in the network. For example, Internet communication using TCP / IP is possible. The communication I / FC 208 is also connected to the printer 107 by Internet communication or the like. A print instruction is issued from the CPU 201 to the printer 107 via the communication I / FC 208.

  Note that the CPU 201 enables display on the display 210 by executing outline font rasterization processing on a display information area in the RAM 203, for example. Further, the CPU 201 enables a user instruction with a mouse cursor (not shown) on the display 210.

  Various programs used by the information processing apparatus 101 of the present invention to execute various processes described later are recorded in the external memory 211 and are executed by the CPU 201 by being loaded into the RAM 203 as necessary. is there. Furthermore, definition files and various information tables used by the program according to the present invention are stored in the external memory 211.

  Next, a functional configuration diagram showing a module configuration of the information processing apparatus 101 will be described with reference to FIG. Note that the module configuration of the apparatus of FIG. 3 is an example, and there are various configuration examples depending on the application and purpose.

  As described above, the information processing apparatus 101 includes a trail recording application. The trail recording application includes an input reception unit 301, a display instruction unit 302, an image storage unit 303, an object detection unit 304, an imaging instruction unit 305, an OCR processing unit 306, an operation determination unit 307, and a display control unit 308.

  The input receiving unit 301 receives infrared information transmitted from the infrared sensor 103. The received infrared information is sent to the object detection unit 304. Also, moving images and still images transmitted from the camera 102 are accepted as image data. The received image data is sent to the image storage unit 303.

  Display instruction unit 302 transmits to projector 104 the result of display control unit 308 controlling PDF data, image data, and the like stored in RAM 203 or external memory 211 of information processing apparatus 101.

  The image storage unit 303 cuts out an area corresponding to an object such as a paper document from the image data sent from the input reception unit 301 based on the information detected by the object detection unit 304, and the cut image data is stored in the RAM 203 or the like. Save in the external memory 211. In addition, the image storage unit 303 transmits the stored image data to the display instruction unit 302.

  The object detection unit 304 detects the type of the object on the projection plane 106 based on the infrared information received by the input reception unit 301. For example, it is detected whether the object on the projection surface 106 is a user's finger (indicator) or a paper document. Moreover, the approach direction of the object can also be detected.

  The shooting instruction unit 305 issues a shooting instruction to the camera 102 when the operation determination unit 307 determines that the user has given a shooting instruction. In addition, when the operation determination unit 307 determines that an operation for leaving the user's trail has been performed, the camera 102 is also instructed to take a picture.

  The OCR processing unit 306 detects a character area of the image data stored in the RAM 203 or the external memory 211 of the information processing apparatus 101 using a technique such as OCR (Optical Character Recognition), and each character in the image data. Identify what character is.

  The operation determination unit 307 determines the position and gesture of the object detected by the object detection unit 304 in the real space. Also, a process associated with a position in the real space and a gesture is determined. For example, the control content for the projected image data is determined, and the shooting instruction for the object placed on the projection surface 106 is determined.

  The display control unit 308 executes enlargement / reduction, display / non-display, movement, and the like of the image data projected on the projection plane 106 according to the result determined by the operation determination unit. For example, when the operation determination unit 307 determines a pinch-in / pinch-out gesture on image data to be enlarged / reduced, the display control unit 308 performs image data enlargement / reduction processing.

  The image conversion unit 309 can convert the image data acquired when the shooting instruction unit 305 gives a shooting instruction into the TIFF format. The image conversion format is not limited to the TIFF format, and can be converted into various formats.

  Next, a series of processing performed by the information processing apparatus 101 in the embodiment of the present invention will be described with reference to the flowchart shown in FIG.

  In step S501, the CPU 201 of the information processing apparatus 101 activates a trail recording application stored in the external memory 211 in response to an instruction from a person (hereinafter referred to as a teller) who explains a financial product or the like to a customer. To do. The trail recording application is an application for recording a trail of a series of flows assuming an explanation of financial products performed at a bank window or the like.

In step S <b> 502, the CPU 201 of the information processing apparatus 101 displays an ID input acceptance screen for inputting a teller ID and a customer ID on the screen of the display 210. For example, it is a screen like the ID input acceptance screen 601 shown in FIG. The teller ID input field 602 of the ID input acceptance screen 601 is a field in which a teller ID is input. A customer ID input column 603 is a column in which a customer ID is input. The teller ID is managed in the teller ID management table 700 of FIG. 7, and information such as the teller ID and the name of the teller is linked. The teller ID management table 700 includes a teller ID 701 and a teller name 702. The teller ID 701 is an ID for uniquely determining a teller, and the teller name 702 represents the teller name. The customer ID is managed in the customer ID table 800 of FIG. 8, and the customer ID is associated with information such as the customer's name. The customer ID table 800 includes a customer ID 801 and a customer name 802. The customer ID 801 is an ID for uniquely determining a customer, and the customer name 802 represents the name of the customer.
In step S503, the CPU 201 of the information processing apparatus 101 accepts input of the teller and customer IDs on the ID input acceptance screen 601 displayed in step S502. When the OK button 604 is pressed, the process proceeds to the next step S504. When the cancel button 605 is pressed, the activation of the trail recording application is terminated, and this series of processing is terminated.

  In step S <b> 504, the CPU 201 of the information processing apparatus 101 receives a projection instruction from a teller in order to project document data and image data from the projector 104 onto the projection surface 106. Specifically, the selection of document data or image data to be projected is accepted from the teller. For example, it is PDF data for explaining a financial product. The file format of the document may be other than the PDF format.

  In step S505, the CPU 201 of the information processing apparatus 101 issues a projection instruction to project the PDF data selected from the teller to the projector 104 in accordance with the projection instruction from the teller received in step S504. Details of the projection instruction processing will be described later with reference to FIG.

  In step S <b> 506, the CPU 201 of the information processing apparatus 101 receives from the infrared sensor 103 infrared information (hereinafter referred to as infrared information) that is irradiated from the infrared sensor 103 onto the projection surface 106 and bounces back. A portion where the infrared ray does not rebound from the projection surface 106 is recognized as an object.

  In step S507, the CPU 201 of the information processing apparatus 101 determines whether an object has been detected based on the infrared information received in step S506 (motion detection unit). The object here refers to, for example, a user's finger or a paper document. If an object is detected, the process proceeds to step S508. If no object is detected, the process proceeds to step S512.

  In step S508, the CPU 201 of the information processing apparatus 101 specifies whether the object detected in step S507 is a finger or a paper document (object specifying unit). More specifically, the type of object is specified from the infrared information received by the infrared sensor 103. For example, when it is recognized that the shape of the object is a shape in which a rod-shaped object protrudes from a spherical object, the protruding part is specified as a finger. If the object determined by the infrared information received by the infrared sensor 103 is recognized as a rectangle, the object is specified as a paper document.

  In step S509, the CPU 201 of the information processing apparatus 101 determines whether the detected object leaves a trail based on the type of object determined in step S508 (object determination unit). For example, if it is determined that the object is a finger, it is determined that a trail is to be left, and the process proceeds to the touch determination process in step S510. If it is determined that the object is a paper document, it is determined that no trail is left, and the process proceeds to a paper document capturing process in step S511. Such a determination is made because it is desired to record, as a trail, what operation a user (teller or customer) using the projection system 100 has performed on the projection plane 106.

  In step S510, the CPU 201 of the information processing apparatus 101 determines whether the finger is touching the projected document data or image data (hereinafter, “projected image”). If it is determined that the finger is touched, the projection is performed. A trail is left by capturing image data of a finger touching the image. Details of the touch determination processing will be described in detail later with reference to FIG.

  In step S511, the CPU 201 of the information processing apparatus 101 captures a paper document placed on the projection surface 106 and captures it as image data. More specifically, it captures documents necessary for financial product contracts, such as resident's cards and family register copies brought by customers. Details of the paper document capturing process will be described in detail later with reference to FIG.

  In step S512, the CPU 201 of the information processing apparatus 101 determines whether there is an instruction to end projection from the user. When an instruction to end projection is received from the user, the projection is ended. If it is not determined that the end of projection has been received, the process returns to step S506.

  In step S513, the CPU 201 of the information processing apparatus 101 performs an application termination process in response to an instruction from the user.

  Next, the projection processing in step S505 described above will be described with reference to FIG.

  In step S901, the CPU 201 of the information processing apparatus 101 determines whether or not there is a projection instruction from the teller in step S504 described above. If there is no instruction, the projection processing ends. If a projection instruction is accepted, the process proceeds to step S902.

  In step S <b> 902, the CPU 201 of the information processing apparatus 101 acquires document data or image data (in this case, PDF data, for example) that is instructed to be projected from the teller in step S <b> 504 and stores it in the RAM 203. After obtaining the PDF data (electronic data), the process proceeds to step S903.

  In step S <b> 903, the CPU 201 of the information processing apparatus 101 instructs the projector 104 to project the PDF data acquired in the RAM 203.

  In step S <b> 904, the projector 104 receives a projection instruction from the information processing apparatus 101.

  In step S905, the projector 104 projects the PDF data (electronic data) instructed to be projected onto the projection plane 106 according to the instruction received in step S904. FIG. 10 shows an example of the projection surface 106 obtained as a result of the projection in step S905. The teller 1001 is a person who explains, for example, a financial product to a customer, and activates a trail recording application, explains a product to the customer 1002, and the like. The application form 1003 on the projection surface 106 includes an outline explanation part 1004, a detailed explanation part 1005, a personal information explanation part 1006, and a signature field 1007. When telling the customer 1002 about the product regarding the application form 1003, the teller 1001 explains the outline explanation portion 1004, the detailed explanation portion 1005, and the personal information explanation portion 1006 by using a finger or the like. After receiving the explanation from the teller 1001, the customer 1002 enters the signature in the signature column 1007. The rotation button 1008 is used to rotate the projected document data or image data by 180 degrees when a teller's finger or the like is touched with the rotation button 1008. In FIG. 10, when the rotation button 1008 is pressed, the application form 1003 on the projection plane 106 is rotated 180 degrees. That is, the application form 1003 facing the customer 1002 faces the teller 1001. The rotation button 1008 may be a physical button or a projected button image. This is the end of the description of the projection process.

  Next, the touch determination process in step S510 will be described with reference to FIG.

  In step S <b> 1101, the CPU 201 of the information processing apparatus 101 specifies a projection source document that is a basis of a projection image projected on the projection plane 106. Since the projection source document is the document data or image data selected in step S504 described above, the projection source document is specified based on this. Here, the file name and storage location of the projection image, and the page number of the projection source document being projected are specified. If specified, the process proceeds to step S1102.

  In step S1102, the CPU 201 of the information processing apparatus 101 acquires imaging area information corresponding to the projection image specified in step S1101. The imaging area information is information existing for each predetermined area included in the projection image, and indicates each record stored in the imaging area information table 1200 shown in FIG. More specifically, the area ID 1201, name 1202, storage location 1203, page 1204, imaging position area 1205, and attribute 1206 are indicated.

  The imaging area information table 1200 is a table stored in the external memory 211 of the information processing apparatus 101. The imaging area information table 1200 includes an area ID 1201, a name 1202, a storage location 1203, a page 1204, an imaging position area 1205, and an attribute 1206. In the area ID 1201, an area ID number such as a number is written in each area for storing a trail. The name 1202 describes a specific name of the area for storing the trail. A storage location 1203 describes a location where the PDF data to be projected is stored. In page 1204, the number of pages of the document to be projected is described. In the imaging position area 1205, position information for storing a trail on the PDF data to be projected is described. The left XY coordinates separated by commas indicate the upper left corner of the rectangle. The right XY coordinates separated by commas indicate the lower right corner of the rectangle. A rectangle consisting of these two points is used as an area for imaging. In addition, although the area | region to image is made into the rectangle in a present Example, it may be circular shape or strip | belt shape. An attribute 1206 describes the attribute of the area for storing the trail. In this embodiment, there are attributes of explanation and consent, but other attributes may be used.

  For example, when acquiring the imaging area information of the application form 1003 in FIG. 10, the file name, storage location, and projected page number of the application form 1003 are specified in step S1101. Based on the identified information and the storage location 1203, a necessary record is identified and acquired. When the file name of the application form 1003 is “A.pdf”, the storage location is “D: \ document \ data \”, and the projected page number is “2”, the area ID 1201 shown in FIG. It corresponds to “1” to “4”. Therefore, these four records are acquired as imaging area information.

  In step S <b> 1103, the CPU 201 of the information processing apparatus 101 receives infrared information transmitted from the infrared sensor 103.

  In step S1104, the CPU 201 of the information processing apparatus 101 analyzes the position of the user's hand on the projection plane 106 based on the infrared information received in step S1103.

  The method for analyzing the position of the user's hand is to use a conventional technique, and the infrared sensor 103 analyzes a shape in which infrared bounce cannot be detected. By analyzing the shape, it is possible to specify which part the user's finger is pointing to.

  In step S1105, the CPU 201 of the information processing apparatus 101 specifies who is the analyzed hand based on the position of the user's hand analyzed in step S1104 (user specifying unit). More specifically, it is specified from the incident direction of the hand and the direction of the hand. For example, in FIG. 10, when the rotation button 1008 comes in from a certain direction, it is specified as a teller's hand, and otherwise, it is specified as a customer's hand. The method of identifying who is the hand is not limited to this. Information that can identify the user may be given to the user's hand, and this may be captured by the camera 102 and analyzed to identify the captured image data.

  In step S1106, the CPU 201 of the information processing apparatus 101 determines whether the user's hand has touched the area indicating the imaging position area 1205 included in the imaging area information acquired in step S1102. If it is determined that the user's hand has touched the imaging position area 1205, the process proceeds to step S1107. If it is not determined that the user's hand has touched the imaging position area 1205, the touch determination process ends. FIG. 13 is a diagram of an example in which the teller is touching the outline explanation portion 1004 that is the imaging position area 1205. If the teller's hand 1301 touches the outline explanation portion 1004, it means that the teller is explaining the outline explanation portion 1004. In other words, it is necessary to take a trail to prove that the summary explanation portion 1004 has been explained to the customer. Therefore, in step S1106, if the area indicating the imaging position area 1205 is touched in this way, the process proceeds to step S1107.

  In step S1107, the CPU 201 of the information processing apparatus 101 determines whether the attribute of the area indicating the touched imaging position area 1205 is an area obligated to explain to the customer from the teller (hereinafter referred to as an explanation trail area), whether the customer has a contract, etc. It is determined whether it is an area for indicating the intent of consent (hereinafter referred to as consent trail area). If an attribute 1206 in the imaging area information acquired in step S1102 corresponding to the touched imaging position area 1205 is specified, the attribute of the trail of the touched imaging position area 1205 is specified, and determination is made according to the specified attribute. Good. If it is determined that the area indicating the imaging position area 1205 is an explanation trail area, the process proceeds to step S1108. If it is determined that the area indicating the imaging position area 1205 is the consent trail area, the process proceeds to step S1110. By determining the attribute of the touched area as described above, it is possible to change the imaging condition according to the attribute indicated by the touched area.

  For example, since the attribute 1206 corresponding to the area ID 1201 “1” of the outline explanation portion 1004 in FIG. 13 is “description”, the outline explanation portion 1004 is specified as the explanation trail area. Therefore, the process proceeds to step S1108.

  On the other hand, for example, FIG. 16 is a diagram in which the customer 1002 signs the signature field 1007 with the customer's hand 1602. In the projection system 100, when a predetermined area such as the signature field 1007 is moved while being touched with a finger, the locus can be drawn. If the CPU 201 of the information processing apparatus 101 determines that the user's finger is touching a predetermined area such as the signature field 1007 based on the infrared information transmitted from the infrared sensor 103, the touch is continued. Meanwhile, the trajectory of the touch is expressed by a line element. By using such a mechanism, a sign can be performed by touching the projection surface 106.

  As described above, when the signature field 1007 is touched to sign, step S1108 for determining whether the hand is the customer's hand or the hand of the teller and step S1109 for determining the orientation of the projected image are performed. Instead, the process immediately proceeds to step S1110. In the consent trail area, the teller may sign consent on behalf of the customer. That is, when the customer does not indicate an intent to consent, the teller signs instead. In such a case, if the trail is preserved, it is possible to prevent the illegal act of the teller and to protect the customer even if it becomes a dispute. For example, in FIG. 17, the teller signs the signature field 1007 which is the consent trail area, and the direction of the projected image is the teller direction. In the case of FIG. 18, the teller is also signed, and the direction of the projected image is for the customer. Even in such a case, if the hand is touched in the signature field 1007, the signature field 1007 is a consent trail area, and the process proceeds to step S1110.

  In step S1108, the CPU 201 of the information processing apparatus 101 determines whether the touched hand is a teller's hand. If the person in the hand identified in step S1105 is a teller, the process proceeds to step S1109. If the person identified in step S1105 is not a teller, the touch determination process is terminated. For example, FIG. 14 is a diagram in which a customer 1002 touches a summary explanation portion 1004 that is an explanation trail area explained by the teller 1001. In this case, since it is not a touch by the teller 1001, the touch determination process is terminated without saving the trail. The explanation trail area only needs to guarantee that the teller explained to the customer. Even when the customer touches the explanation trail area, if the trail is saved, the capacity of the external memory 211 or the like is compressed. Therefore, the trail is taken only when touched by the teller.

  In step S <b> 1109, the CPU 201 of the information processing apparatus 101 determines whether the projected image is facing the customer or the teller. For example, when document data or image data selected from the teller is projected toward the customer, if the rotation button 1008 is pressed, it can be seen that the projected image faces the teller. On the other hand, if the rotation button 1008 is not pressed, it can be seen that the projected image faces the customer. If it is determined that the projected image is facing the customer, the process proceeds to step S1110. If it is not determined that the projected image is facing the customer, the touch determination process is terminated. For example, in FIG. 15, the projected image is directed to the teller, not the customer. Even if the teller explained in this state, it cannot be said that the customer explained normally because the customer cannot read the projected image. Therefore, in this case, the touch determination process is terminated without saving the trail. Thus, in the case of the explanation trail area, the trail is saved when the teller touches the explanation trail area and the projection image faces the customer. If the teller does not explain it in the correct way, there will be no trail, which can be evidence of whether or not it has been explained.

  In step S1110, the CPU 201 of the information processing apparatus 101 issues an imaging instruction without digital imaging sound to the camera 102 (imaging instruction transmission unit / imaging sound control unit). In this embodiment, the elimination of the imaging sound means that the output of the imaging sound is eliminated, but the imaging sound may be of a volume that cannot be heard. In addition, the camera is instructed to perform imaging without imaging sound, but the camera does not need to output imaging sound (imaging sound output means), and a speaker connected to the information processing apparatus 101 outputs imaging sound. May be.

  In step S <b> 1111, the camera 102 receives an imaging instruction without imaging sound transmitted from the information processing apparatus 101.

  In step S <b> 1112, the camera 102 performs imaging that does not emit imaging sound in accordance with the received imaging instruction. The image pickup sound here is a digital image pickup sound. The imaging sound is not emitted when switching between imaging that produces imaging sound and imaging that does not produce imaging sound, or the volume of the imaging sound itself may be reduced. The method of not producing the imaging sound is not limited to the above. By not producing digital imaging sound, it is possible to avoid recognizing that the customer is taking a trail.

  In step S1113, the camera 102 transmits captured image data to the information processing apparatus.

  In step S1114, the CPU 201 of the information processing apparatus 101 receives image data transmitted from the camera.

  In step S1115, the CPU 201 of the information processing apparatus 101 converts the received image data into a trail ID, a captured date (date and time), a customer ID, a teller ID, a region ID, and image data as a trail for each customer. Save to. As an example of saving as a trail, there is a trail saving table 1900 of FIG. The trail ID column 1901 stores an ID number for uniquely determining a trail. The year / month / day column 1902 stores the date and time when the image was taken to save the trail. The imaged user field 1903 stores information that is specified in step S1105, whether the imaged user is a teller or a customer. In the captured customer / teller ID column 1904, the ID of the teller or customer input in step S503 is stored. The image data column 1905 stores information indicating the folder location where the captured image data is stored. In this embodiment, the TIFF format is adopted as the image storage format, but other formats may be used. By managing each customer in the table format in this way, it is easy to search for trails, and it can be a useful resource in the event of a dispute with the customer. Since image data indicating a finger remains in the projected document, it is possible to accumulate data that can be secured as a trail.

  Next, the paper document capturing process will be described with reference to FIG.

  In step S2001, the CPU 201 of the information processing apparatus 101 receives infrared information sent from the infrared sensor 103.

  In step S2002, the CPU 201 of the information processing apparatus 101 analyzes the infrared information received in step S2001, and analyzes the movement of the paper document.

  In step S2003, the CPU 201 of the information processing apparatus 101 determines whether the paper document analyzed in step S2002 can be imaged. For example, when it is determined that the paper document has not moved for a certain period of time, it is determined that imaging is possible. If it is determined that imaging is possible, the process proceeds to step S2004. Note that the determination that imaging is possible may be an imaging instruction from the user.

  In step S2004, the CPU 201 of the information processing apparatus 101 determines the position of the projection image projected on the projection surface 106 when it is determined in step S2003 that the paper document is placed and the movement has stopped for a certain period of time, Process to save the state. For example, there is a display menu 2003 displayed in FIG. 21, and these states are stored. When the storage process ends, the process moves to step S2005.

  In step S2005, the CPU 201 of the information processing apparatus 101 instructs the projector 104 to erase what is being projected. It is only necessary to be able to control so that what is currently projected on the projection surface 106 is not projected. This is to prevent the projected image from being captured when a paper document is captured in the process described later. Therefore, it is sufficient that projection is not performed on the placed paper document, and some information may be projected in other areas. For example, when a paper document is placed on the display menu 2003 in FIG. 21, all projections may be erased, or only a projection of a portion where the paper document and the display menu 2003 overlap may be erased.

  In step S2006, the projector 104 receives a projection non-display instruction instructed from the information processing apparatus 101.

  In step S2007, the projector 104 erases what is currently projected according to the received projection non-display instruction.

  In step S2008, the CPU 201 of the information processing apparatus 101 instructs the camera 102 to perform imaging with a digital imaging sound (imaging sound output unit). The instruction for imaging with digital imaging sound may be an instruction for outputting imaging sound to a camera that originally does not emit sound.

  In step S2009, the camera 102 receives an imaging instruction with imaging sound transmitted from the information processing apparatus 101.

  In step S2010, the camera 102 performs imaging with digital imaging sound according to the received instruction. By performing imaging with imaging sound, it is possible to clearly recognize that documents such as a customer's resident's card and family register copy are taken in as image data.

  In step S <b> 2011, the camera 102 transmits image data obtained by imaging to the information processing apparatus 101.

  In step S2012, the CPU 201 of the information processing apparatus 101 receives the image data transmitted from the camera 102.

  In step S2013, the CPU 201 of the information processing apparatus 101 stores the received image data as paper document capture data.

  In step S2014, the CPU 201 of the information processing apparatus 101 reads out the state saved in step S2004 and returns the projection state to the projector 104 to resume the projection, and resumes the projection.

  In step S2015, the projector 104 receives a projection display instruction from the information processing apparatus 101.

  In step S2016, the state saved in step S2004 is projected on the projector 104 based on the received projection display instruction.

  As described above, according to the present embodiment, it is possible to control the imaging sound of the imaging device when taking a trail.

  Next, a second embodiment of the present invention will be described. In the first embodiment, when the attribute of the touched area is a consent trail, an imaging instruction is performed without determining whether the touched hand is a customer or a teller and determining the orientation of the projected document. It was. In the second embodiment, even in the case of the consent trail, the above determination is made, so that it is possible to leave a trail that the customer has surely signed in the area of the consent trail. This makes it possible to leave only an appropriate trail without leaving a useless trail such as when a teller explains a sign to a customer.

  In the second embodiment, the system configuration, hardware configuration, functional configuration, screen example, and table are the same as those in the first embodiment, and FIG. 9 is modified to FIG. 24 and FIG. 11 is modified to FIG. .

  The projection process of the second embodiment will be described with reference to the flowchart of FIG.

  In step S2401, the CPU 201 of the information processing apparatus 101 acquires document data or image data (projection document) (in this case, for example, PDF data) that is instructed to be projected from the teller in step S504, and stores it in the RAM 203. The file name of the projection document whose selection is accepted in step S504 is stored in the projection document 2301 of the projection table 2300. In the projection table 2300 shown in FIG. 23, a projection document 2301 and a rotation flag 2302 are stored. The projection document 2301 stores the file name of the projection document that has been selected in step S504. The rotation flag 2302 is a flag indicating the orientation of the projected document. When the rotation flag 2302 is “0”, the projection document is directed to the customer. When the rotation flag 2302 is “1” (when the flag is set), the projection is projected as the teller orientation. An instruction to project a document is shown. After obtaining the PDF data in step S2401, the process proceeds to step S2402.

  In step S2402, the CPU 201 of the information processing apparatus 101 determines whether the rotation flag 2302 of the projection table 2300 is “1” (whether it is standing). If it is determined that the rotation flag 2302 is “1”, the process advances to step S2303 to rotate the projection document. If it is determined that the rotation flag 2302 is not “1”, the process proceeds to step S2304.

  In step S2403, the CPU 201 of the information processing apparatus 101 performs a process of rotating the orientation of the projection document 2301 stored in the RAM 203 by 180 degrees. As a result, the projection document that was the orientation of the teller becomes the customer's orientation, and the projection document that was the customer's orientation becomes the teller's orientation.

  In step S2404, the CPU 201 of the information processing apparatus 101 instructs the projector 104 to project PDF data (electronic data) stored in the RAM 203.

  In step S <b> 2405, the projector 104 receives a projection instruction from the information processing apparatus 101.

  In step S2406, the projector 104 projects the PDF data instructed to be projected onto the projection plane 106 according to the instruction received in step S2403. FIG. 10 shows an example of the projection surface 106 obtained as a result of projection according to a series of this process. The detailed description of FIG. 10 is as described above. This is the end of the description of the projection process.

  Next, the touch determination process in step S510 will be described with reference to FIG.

  In step S <b> 2501, the CPU 201 of the information processing apparatus 101 specifies a projection document that is a basis of the projection image projected on the projection plane 106. Since the projected document is the document data or image data selected in step S504 described above, the projection document is specified based on this. Here, the file name and storage location of the projection image, and the page number of the projection source document being projected are specified. If specified, the process proceeds to step S2502.

  In step S2502, the CPU 201 of the information processing apparatus 101 acquires imaging area information corresponding to the projection image specified in step S2501. The imaging area information is information existing for each predetermined area included in the projection image, and indicates each record stored in the imaging area information table 1200 shown in FIG. More specifically, the area ID 1201, name 1202, storage location 1203, page 1204, imaging position area 1205, and attribute 1206 are indicated.

  The detailed description of the imaging area information table 1200 is as described above.

  For example, when acquiring the imaging area information of the application form 1003 in FIG. 10, the file name, storage location, and projected page number of the application form 1003 are specified in step S2501. Based on the identified information and the storage location 1203, a necessary record is identified and acquired. When the file name of the application form 1003 is “A.pdf”, the storage location is “D: \ document \ data \”, and the projected page number is “2”, the area ID 1201 shown in FIG. It corresponds to “1” to “4”. Therefore, these four records are acquired as imaging area information.

  In step S2503, the CPU 201 of the information processing apparatus 101 analyzes the position of the user's hand on the projection plane 106 based on the infrared information received in step S506. The method for analyzing the position of the user's hand is to use a conventional technique, and the infrared sensor 103 analyzes a shape in which infrared bounce cannot be detected. By analyzing the shape, it is possible to specify which part the user's finger is pointing to.

  In step S2504, the CPU 201 of the information processing apparatus 101 specifies who is the analyzed hand based on the position of the user's hand analyzed in step S2503 (user specifying unit). More specifically, it is specified from the approach direction of the hand and the direction of the hand. For example, in FIG. 10, when the rotation button 1008 comes in from a certain direction, it is specified as a teller's hand, and otherwise, it is specified as a customer's hand. The method of identifying who is the hand is not limited to this. Information that can identify the user may be given to the user's hand, and this may be captured by the camera 102 and analyzed to identify the captured image data.

  In step S2505, the CPU 201 of the information processing apparatus 101 determines whether or not the user's hand has touched the area indicated by the imaging position area 1205 included in the imaging area information acquired in step S2502 (first determination unit). If it is determined that the user's hand has touched the imaging position area 1205, the process proceeds to step S2506. If it is not determined that the user's hand has touched the imaging position area 1205, the process proceeds to step S2515. FIG. 13 is a diagram of an example in which the teller is touching the outline explanation portion 1004 that is the imaging position area 1205. If the teller's hand 1301 touches the outline explanation portion 1004, it means that the teller is explaining the outline explanation portion 1004. In other words, it is necessary to take a trail to prove that the summary explanation portion 1004 has been explained to the customer. Therefore, in step S2505, if the area indicating the imaging position area 1205 is touched in this way, the process proceeds to step S2506. If it is determined that the area indicating the imaging position area 1205 is not touched, the process proceeds to step S2515 to determine whether the rotation button has been pressed.

  In step S2506, the CPU 201 of the information processing apparatus 101 determines whether the attribute of the area indicating the touched imaging position area 1205 is an area obligated to explain to the customer from the teller (hereinafter referred to as an explanation trail area) It is determined whether it is an area for indicating the intention of consent (hereinafter referred to as consent trail area) (second determination means). If an attribute 1206 in the imaging area information acquired in step S2502 corresponding to the touched imaging position area 1205 is specified, the trail attribute of the touched imaging position area 1205 is specified, and determination is made according to the specified attribute. Good. If it is determined that the area indicating the imaging position area 1205 is an explanation trail area, the process proceeds to step S2507. If it is determined that the area indicating the imaging position area 1205 is the consent trail area, the process proceeds to step S2517. By determining the attribute of the touched area as described above, it is possible to change the imaging condition according to the attribute indicated by the touched area.

  For example, since the attribute 1206 corresponding to the area ID 1201 “1” of the outline explanation portion 1004 in FIG. 13 is “description”, the outline explanation portion 1004 is specified as the explanation trail area. Therefore, the process proceeds to step S2507.

  On the other hand, for example, FIG. 16 is a diagram in which the customer 1002 signs the signature field 1007 with the customer's hand 1602. In the projection system 100, when a predetermined area such as the signature field 1007 is moved while being touched with a finger, the locus can be drawn. If the CPU 201 of the information processing apparatus 101 determines that the user's finger is touching a predetermined area such as the signature field 1007 based on the infrared information transmitted from the infrared sensor 103, the touch is continued. Meanwhile, the trajectory of the touch is expressed by a line element. By using such a mechanism, a sign can be performed by touching the projection surface 106.

  In the first embodiment, when the signature field 1007 is touched, the step of determining whether it is the customer's hand or the hand of the teller and the step of determining the direction of the projected image are performed immediately without performing the steps. The process proceeds to step S1109. In the second embodiment shown in FIG. 26, the process does not proceed to the immediate step S2509, but determines whether the direction of the projected document is the customer's hand. As a result, it is possible to leave a trail only for the customer, so that the storage capacity of the external memory 211 and the like can be saved. In addition, there is an effect that a teller does not leave a useless trail when explaining a signing method to a customer.

  The determinations in steps S2507 to 2608 and steps S2517 to 2618 are performed based on the condition table 2200 shown in FIG. The condition table 2200 stores an area attribute 2201, a rotation flag 2202, a specified hand 2203, and processing content 2204. An area attribute 2201 indicates the attribute of the area determined in step S2506. A rotation flag 2202 indicates the flag of the rotation flag 2302 of the projection table 2300. The identified hand 2203 schematically shows whether it is the teller's hand or the customer's hand identified in step S2504. The processing content 2204 describes the content of processing for determining whether or not to take an image. For example, if the region attribute 2201 is an explanation trail region, the rotation flag 2202 is 0, and the identified hand 2203 is a teller, the processing content 2204 is imaged.

  In step S2507, the CPU 201 of the information processing apparatus 101 determines whether the projected image is facing the customer or the teller. For example, if the rotation button 1008 is pressed and the rotation flag 2302 of the projection table 2300 is set in step S2516, it can be understood that the projection image faces the teller. On the other hand, if the rotation flag 2202 is not flagged, it can be seen that the projected image faces the customer. If it is determined that the flag is set in the rotation flag 2202, the process proceeds to step S2508. If it is determined that the rotation flag 2202 is not flagged, the touch determination process is terminated because the projected document is in the teller orientation. For example, in FIG. 15, the projected image is directed to the teller, not the customer. Even if the teller explained in this state, it cannot be said that the customer explained normally because the customer cannot read the projected image. Therefore, in this case, the touch determination process is terminated without saving the trail.

  In step S2508, the CPU 201 of the information processing apparatus 101 determines whether the touched hand is a teller's hand. If the person in the hand identified in step S2504 is a teller, the process proceeds to step S2509. If the person identified in step S2504 is not a teller, the touch determination process is terminated. For example, FIG. 14 is a diagram in which a customer 1002 touches a summary explanation portion 1004 that is an explanation trail area explained by the teller 1001. In this case, since it is not a touch by the teller 1001, the touch determination process is terminated without saving the trail. The explanation trail area only needs to guarantee that the teller explained to the customer. Even when the customer touches the explanation trail area, if the trail is saved, the capacity of the external memory 211 or the like is compressed. Therefore, the trail is taken only when touched by the teller. Thus, in the case of the explanation trail area, the trail is saved when the teller touches the explanation trail area and the projection image faces the customer. If the teller does not explain in the correct way, there will be no trail, which is evidence that it has been explained.

  In step S <b> 2509, the CPU 201 of the information processing apparatus 101 issues an imaging instruction without digital imaging sound to the camera 102 (imaging instruction unit / imaging sound output unit). In this embodiment, the image pickup sound is eliminated, but the image pickup sound may be of a volume that cannot be heard. In addition, although the camera is instructed to perform imaging without imaging sound, the camera does not have to output imaging sound, and a speaker connected to the information processing apparatus 101 may output imaging sound.

  In step S <b> 2510, the camera 102 receives an imaging instruction without imaging sound transmitted from the information processing apparatus 101.

  In step S <b> 2511, the camera 102 performs imaging without producing an imaging sound in accordance with the received imaging instruction. The image pickup sound here is a digital image pickup sound. The imaging sound is not emitted when switching between imaging that produces imaging sound and imaging that does not produce imaging sound, or the volume of the imaging sound itself may be reduced. The method of not producing the imaging sound is not limited to the above. By not producing digital imaging sound, it is possible to avoid recognizing that the customer is taking a trail.

  In step S2512, the camera 102 transmits captured image data to the information processing apparatus.

  In step S2513, the CPU 201 of the information processing apparatus 101 receives image data transmitted from the camera.

  In step S2514, the CPU 201 of the information processing apparatus 101 stores the received image data for each customer as a trail by collecting the trail ID, the imaged date, the customer ID, the teller ID, the region ID, and the image data. (Image data storage means). As an example of saving as a trail, there is a trail saving table 1900 of FIG. The detailed description of the trail storage table 1900 is as described above. By managing each customer in the table format in this way, it is easy to search for trails, and it can be a useful resource in the event of a dispute with the customer. Since the image data actually pointing the finger remains in the projected document, it is possible to accumulate data that can be secured as a trail.

  In step S2515, the CPU 201 of the information processing apparatus 101 determines whether or not the rotation button 1008 arranged on the projection plane 106 has been touched. By accepting the touch of the rotation button 1008, it can be determined that there has been an instruction to rotate the projection document from the user. If it is determined that the rotation button 1008 has been touched, the process proceeds to step S2516. If it is determined that the rotation button 1008 has not been touched, the process ends.

  In step S2516, the CPU 201 of the information processing apparatus 101 inverts the rotation flag 2302 of the projection table 2300. For example, “0” to “1”, “1” to “0”, and so on. After inverting the flag, the process proceeds to step S2402.

  In step S2517, the CPU 201 of the information processing apparatus 101 determines whether the projected image is facing the customer or the teller. For example, if the rotation button 1008 is pressed and the rotation flag 2302 of the projection table 2300 is set in step S2516, it can be understood that the projection image faces the teller. On the other hand, if the rotation flag 2202 is not flagged, it can be seen that the projected image faces the customer. If it is determined that the rotation flag 2202 is flagged, the process advances to step S2509. If it is determined that the rotation flag 2202 is not flagged, the touch determination process is terminated because the projected document is in the teller orientation.

  In step S2518, the CPU 201 of the information processing apparatus 101 determines whether the touched hand is the customer's hand. If the person of the hand specified in step S2504 is a customer, the process proceeds to step S2509. If the person in the hand specified in step S2504 is not a customer, the touch determination process is terminated. By performing the determinations in steps S2517 and 2518, it is possible to leave only an appropriate trail without leaving a useless trail such as when a teller explains a sign to a customer.

  As described above, even in the case of consent trail, by making the above determination, it is possible to leave a trail that the customer has surely signed in the area of the consent trail. For example, it is possible to leave only an appropriate trail without leaving a useless trail such as when a teller explains a sign to a customer.

  Next, a third embodiment of the present invention will be described. In the first embodiment, the mechanism for controlling the output of the imaging sound by the imaging instruction when saving the trail and the imaging instruction when capturing the paper document has been described. In the third embodiment, the type of imaging sound is switched between an imaging instruction for saving a trail and an imaging instruction for capturing a paper document. As a result, it is possible to notify the user of the intention of imaging.

  In the third embodiment, the system configuration, functional configuration, screen example, and table are the same as in the first embodiment, and FIG. 9 is modified to FIG. 24, FIG. 11 is modified to FIG. 27, and FIG. It is.

  The hardware configuration of the third embodiment will be described with reference to FIG.

  26 is a configuration in which a sound controller 212 and a speaker 213 are newly added to the configuration of FIG. 2, and description of common items is omitted.

  The sound controller 212 is a controller that processes an audio signal, and controls audio output from the speaker 213.

  The speaker 213 is connected to the information processing apparatus 101 through a speaker cable, and outputs a signal output from the sound controller 212 as sound.

  Next, the touch determination process of the third embodiment will be described with reference to FIG.

  In step S2701, the CPU 201 of the information processing apparatus 101 specifies a projection document that is a basis of the projection image projected on the projection plane. Since the projected document is the document data or image data selected in step S504 described above, the projection document is specified based on this. Here, the file name and storage location of the projection image, and the page number of the projection source document being projected are specified. If specified, the process proceeds to step S2702.

  In step S2702, the CPU 201 of the information processing apparatus 101 acquires imaging area information corresponding to the projection image specified in step S2701. The imaging area information is information existing for each predetermined area included in the projection image, and indicates each record stored in the imaging area information table 1200 shown in FIG. More specifically, the area ID 1201, name 1202, storage location 1203, page 1204, imaging position area 1205, and attribute 1206 are indicated.

  The imaging area information table 1200 is a table stored in the external memory 211 of the information processing apparatus 101. The detailed description of the imaging area information table 1200 is as described above.

  For example, when acquiring the imaging area information of the application form 1003 in FIG. 10, the file name, storage location, and projected page number of the application form 1003 are specified in step S2701. Based on the identified information and the storage location 1203, a necessary record is identified and acquired. When the file name of the application form 1003 is “A.pdf”, the storage location is “D: \ document \ data \”, and the projected page number is “2”, the area ID 1201 shown in FIG. It corresponds to “1” to “4”. Therefore, these four records are acquired as imaging area information.

  In step S2703, the CPU 201 of the information processing apparatus 101 analyzes the position of the user's hand on the projection plane 106 based on the infrared information received in step S506. The method for analyzing the position of the user's hand is to use a conventional technique, and the infrared sensor 103 analyzes a shape in which infrared bounce cannot be detected. By analyzing the shape, it is possible to specify which part the user's finger is pointing to.

  In step S2704, the CPU 201 of the information processing apparatus 101 specifies who is the analyzed hand based on the position of the user's hand analyzed in step S2703 (user specifying unit). More specifically, it is specified from the approach direction of the hand and the direction of the hand. For example, in FIG. 10, when the rotation button 1008 comes in from a certain direction, it is specified as a teller's hand, and otherwise, it is specified as a customer's hand. The method of identifying who is the hand is not limited to this. Information that can identify the user may be given to the user's hand, and this may be captured by the camera 102 and analyzed to identify the captured image data.

  In step S2705, the CPU 201 of the information processing apparatus 101 determines whether the user's hand has touched the area indicated by the imaging position area 1205 included in the imaging area information acquired in step S2702 (first determination unit). If it is determined that the user's hand has touched the imaging position area 1205, the process proceeds to step S2706. If it is not determined that the user's hand has touched the imaging position area 1205, the process proceeds to step S2720. FIG. 13 is a diagram of an example in which the teller is touching the outline explanation portion 1004 that is the imaging position area 1205. If the teller's hand 1301 touches the outline explanation portion 1004, it means that the teller is explaining the outline explanation portion 1004. In other words, it is necessary to take a trail to prove that the summary explanation portion 1004 has been explained to the customer. Therefore, in step S2705, if the area indicating the imaging position area 1205 is touched in this way, the process proceeds to step S2706. If it is determined that the area indicating the imaging position area 1205 is not touched, the process proceeds to step S2720 to determine whether the rotation button has been pressed.

  In step S <b> 2706, the CPU 201 of the information processing apparatus 101 determines whether the attribute of the area indicating the touched imaging position area 1205 is an area obligated to explain to the customer from the teller (hereinafter, an explanation trail area) It is determined whether it is an area for indicating the intention of consent (hereinafter referred to as consent trail area) (second determination means). If an attribute 1206 in the imaging area information acquired in step S2702 corresponding to the touched imaging position area 1205 is specified, the trail attribute of the touched imaging position area 1205 is specified, and determination is made according to the specified attribute. Good. If it is determined that the area indicating the imaging position area 1205 is an explanatory trail area (explanatory operation), the process proceeds to step S2707. If it is determined that the area indicating the imaging position area 1205 is the consent trail area (consent operation), the process proceeds to step S2711. By determining the attribute of the touched area as described above, it is possible to change the imaging condition according to the attribute indicated by the touched area.

  For example, since the attribute 1206 corresponding to the area ID 1201 “1” of the outline explanation portion 1004 in FIG. 13 is “description”, the outline explanation portion 1004 is specified as the explanation trail area. Therefore, the process proceeds to step S2707.

  On the other hand, for example, FIG. 16 is a diagram in which the customer 1002 signs the signature field 1007 with the customer's hand 1602. In the projection system 100, when a predetermined area such as the signature field 1007 is moved while being touched with a finger, the locus can be drawn. If the CPU 201 of the information processing apparatus 101 determines that the user's finger is touching a predetermined area such as the signature field 1007 based on the infrared information transmitted from the infrared sensor 103, the touch is continued. Meanwhile, the trajectory of the touch is expressed by a line element. By using such a mechanism, a sign can be performed by touching the projection surface 106.

  The determinations in steps S2707 to 2608 and steps S2711 to 2612 are performed based on the condition table 2200 shown in FIG. The condition table 2200 stores an area attribute 2201, a rotation flag 2202, a specified hand 2203, and processing content 2204. An area attribute 2201 indicates the attribute of the area determined in step S2706. A rotation flag 2202 indicates the flag of the rotation flag 2302 of the projection table 2300. The identified hand 2203 schematically indicates whether it is the teller's hand or the customer's hand identified in step S2704. The processing content 2204 describes the content of processing for determining whether or not to take an image. For example, if the region attribute 2201 is an explanation trail region, the rotation flag 2202 is 0, and the identified hand 2203 is a teller, the processing content 2204 is imaged.

  In step S <b> 2707, the CPU 201 of the information processing apparatus 101 determines whether the projected image is facing the customer or the teller. For example, if the rotation button 1008 is pressed and the rotation flag 2302 of the projection table 2300 is set in step S2716, it can be understood that the projection image faces the teller. On the other hand, if the rotation flag 2202 is not flagged, it can be seen that the projected image faces the customer. If it is determined that the rotation flag 2202 is flagged, the process advances to step S2708. If it is determined that the rotation flag 2202 is not flagged, the touch determination process is terminated because the projected document is in the teller orientation. For example, in FIG. 15, the projected image is directed to the teller, not the customer. Even if the teller explained in this state, it cannot be said that the customer explained normally because the customer cannot read the projected image. Therefore, in this case, the touch determination process is terminated without saving the trail.

  In step S2708, the CPU 201 of the information processing apparatus 101 determines whether the touched hand is a teller's hand. If the person of the hand specified in step S2704 is a teller, the process proceeds to step S2709. If the person identified in step S2704 is not a teller, the touch determination process is terminated. For example, FIG. 14 is a diagram in which a customer 1002 touches a summary explanation portion 1004 that is an explanation trail area explained by the teller 1001. In this case, since it is not a touch by the teller 1001, the touch determination process is terminated without saving the trail. The explanation trail area only needs to guarantee that the teller explained to the customer. Even when the customer touches the explanation trail area, if the trail is saved, the capacity of the external memory 211 or the like is compressed. Therefore, the trail is taken only when touched by the teller. Thus, in the case of the explanation trail area, the trail is saved when the teller touches the explanation trail area and the projection image faces the customer. If the teller does not explain in the correct way, there will be no trail, which is evidence that it has been explained.

  In step S2709, the CPU 201 of the information processing apparatus 101 issues an imaging instruction to the camera 102 (imaging instruction unit).

  In step S2710, the CPU 201 of the information processing apparatus 101 determines that it is the explanation trail area in step S2706, and therefore outputs the imaging sound of the explanation area from the speaker 213 to notify that the trail in the explanation area is to be saved. (Imaging sound output means). The imaging sound is determined from the imaging sound data table 2900 shown in FIG. The image pickup sound data table 2900 stores an image pickup scene 2901 and an image pickup sound 2902. The image pickup scene 2901 is a scene picked up in the present embodiment, “paper document capture (capture scene)” “explanation trail area (description) Scene) "" consent trail area (consent scene) ". An imaging sound 2902 is assigned to each of them, and audio data stored in the imaging sound 2902 is output according to each scene. For example, since it is determined in step S2706 that the scene is an explanation trail scene (description scene) and the scene is specified, the image pickup sound 2902 corresponding to the “description trail area” corresponding to the scene is specified as “D: ¥ sound ¥ 002. “wav” is specified, and the imaging sound 2902 is controlled to be output from the speaker 213.

  When the imaging sound is output from the speaker 213, it is desirable that the imaging sound be output so as to match the timing in step S2716 described later. For this reason, a delay time from when an imaging instruction is given in step S2709 until imaging is performed in step S2716 described later is stored in advance or calculated by a conventional technique, and the imaging sound is output after waiting for the delay time. What should I do?

  In step S2711, the CPU 201 of the information processing apparatus 101 determines whether the projected image is facing the customer or the teller. For example, if the rotation button 1008 is pressed and the rotation flag 2302 of the projection table 2300 is set in step S2716, it can be understood that the projection image faces the teller. On the other hand, if the rotation flag 2202 is not flagged, it can be seen that the projected image faces the customer. If it is determined that the flag is not set in the rotation flag 2202, the process proceeds to step S2712. If it is determined that the rotation flag 2202 is flagged, the touch determination process is terminated because the projected document is in the teller orientation.

  In step S2712, the CPU 201 of the information processing apparatus 101 determines whether the touched hand is a customer's hand. If the person of the hand specified in step S2704 is a customer, the process proceeds to step S2713. If the person identified in step S2704 is not a customer, the touch determination process is terminated.

  In step S <b> 2713, the CPU 201 of the information processing apparatus 101 issues an imaging instruction to the camera 102.

  In step S2714, the CPU 201 of the information processing apparatus 101 specifies the imaging sound in the explanation area to notify that the trail of the consent area is to be stored, and outputs the imaging sound from the speaker 213 (imaging sound output unit). As in step S2710, the image pickup sound is determined from the image pickup sound data table 2900 shown in FIG. When the imaging sound is output from the speaker 213, it is desirable that the imaging sound be output so as to match the timing in step S2716 described later. For this reason, a delay time from when an imaging instruction is given in step S2713 until imaging is performed in step S2716 described later is stored in advance or calculated by a conventional technique, and the imaging sound is output after waiting for the delay time. What should I do?

  In step S2715, the camera 102 receives the imaging instruction transmitted from the information processing apparatus 101.

  In step S2716, the camera 102 performs imaging according to the received imaging instruction. In the third embodiment, the camera 102 does not output imaging sound.

  In step S2717, the camera 102 transmits captured image data to the information processing apparatus.

  In step S2718, the CPU 201 of the information processing apparatus 101 receives image data transmitted from the camera.

  In step S2719, the CPU 201 of the information processing apparatus 101 stores the received image data for each customer as a trail by collecting the trail ID, the imaged date, the customer ID, the teller ID, the region ID, and the image data. . As an example of saving as a trail, there is a trail saving table 1900 of FIG. The detailed description of the trail storage table 1900 is as described above. By managing each customer in the table format as described above, it is easy to search for trails, and it becomes a useful resource when a dispute occurs with a customer. Since the image data actually pointing the finger remains in the projected document, it is possible to accumulate data that can be secured as a trail.

  In step S2720, the CPU 201 of the information processing apparatus 101 determines whether or not the rotation button 1008 arranged on the projection surface 106 has been touched. By accepting the touch of the rotation button 1008, it can be determined that there has been an instruction to rotate the projection document from the user. If it is determined that the rotation button 1008 has been touched, the process proceeds to step S2721, and if it is determined that the rotation button 1008 has not been touched, the process ends.

  In step S2716, the CPU 201 of the information processing apparatus 101 inverts the rotation flag 2302 of the projection table 2300. For example, “0” to “1”, “1” to “0”, and so on. After inverting the flag, the process proceeds to step S2402.

  Next, a paper document import process in the third embodiment will be described with reference to FIG.

  In step S2801, the CPU 201 of the information processing apparatus 101 analyzes the infrared information received in step S506, and analyzes the movement of the paper document.

  In step S2802, the CPU 201 of the information processing apparatus 101 determines whether the paper document analyzed in step S2801 can be imaged. For example, when it is determined that the paper document has not moved for a certain period of time, it is determined that imaging is possible. If it is determined that imaging is possible, the process proceeds to step S2803. Note that the determination that imaging is possible may be an imaging instruction from the user. For example, a scan button or the like may be arranged on the projection plane, and it may be determined that imaging can be performed by receiving a press of the scan button.

  In step S2803, the CPU 201 of the information processing apparatus 101 determines the position of the projection image projected on the projection plane 106 when it is determined in step S2802 that the paper document is placed and the movement has ceased for a certain period of time. Process to save the state. For example, there is a display menu 2003 displayed in FIG. 21, and these states are stored. When the storage process ends, the process moves to step S2804.

  In step S2804, the CPU 201 of the information processing apparatus 101 instructs the projector 104 to erase what is being projected. It is only necessary to be able to control so that what is currently projected on the projection surface 106 is not projected. This is to prevent the projected image from being captured when a paper document is captured in the process described later. Therefore, it is sufficient that projection is not performed on the placed paper document, and some information may be projected in other areas. For example, when a paper document is placed on the display menu 2003 in FIG. 21, all projections may be erased, or only a projection of a portion where the paper document and the display menu 2003 overlap may be erased.

  In step S2805, the projector 104 receives a projection non-display instruction instructed from the information processing apparatus 101.

  In step S2806, projector 104 erases the currently projected projection in accordance with the received projection non-display instruction.

  In step S <b> 2807, the CPU 201 of the information processing apparatus 101 issues an imaging instruction to the camera 102.

  In step S <b> 2808, the CPU 201 of the information processing apparatus 101 specifies a captured sound for capturing a paper document, and outputs the captured sound through the speaker 213 (imaging sound output unit). The imaging sound is specified from the imaging sound data table 2900 shown in FIG. Since it is specified that the paper document is captured in step S509, the imaging sound 2902 shown in FIG. 29 is “D: ¥ sound ¥ 001.wav” in which the imaging scene 2901 is “paper document capturing”. Further, similarly to step S2714 described above, it is desirable to output in synchronization with the imaging in step S2810 described later. Therefore, as described above in step S2714, the process waits until imaging is performed, and the imaging sound is output after standby.

  In step S <b> 2809, the camera 102 receives an imaging instruction transmitted from the information processing apparatus 101.

  In step S2810, the camera 102 performs imaging with digital imaging sound according to the received instruction. By performing imaging with imaging sound, it is possible to clearly recognize that documents such as a customer's resident's card and family register copy are taken in as image data.

  In step S <b> 2811, the camera 102 transmits image data obtained by imaging to the information processing apparatus 101.

  In step S2812, the CPU 201 of the information processing apparatus 101 receives the image data transmitted from the camera 102.

  In step S2813, the CPU 201 of the information processing apparatus 101 stores the received image data as paper document capture data.

  In step S2814, the CPU 201 of the information processing apparatus 101 reads the state stored in step S2803 and projects the read projection state on the projector 104 to resume the projection, and resumes the projection.

  In step S2815, projector 104 receives a projection display instruction from information processing apparatus 101.

  In step S2816, the state saved in step S2803 is projected onto projector 104 based on the received projection display instruction.

  As described above, in the third embodiment, the user can be notified of the intention of imaging by switching the type of imaging sound between the imaging instruction when saving the trail and the imaging instruction when capturing the paper document. There is an effect that it becomes possible.

  The present invention can be implemented as a system, apparatus, method, program, storage medium, or the like, and can be applied to a system including a plurality of devices. You may apply to the apparatus which consists of one apparatus.

  Note that the present invention includes a software program that implements the functions of the above-described embodiments directly or remotely from a system or apparatus. The present invention also includes a case where the system or the computer of the apparatus is achieved by reading and executing the supplied program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, or the like.

  Examples of the recording medium for supplying the program include a flexible disk, hard disk, optical disk, magneto-optical disk, MO, CD-ROM, CD-R, and CD-RW. In addition, there are magnetic tape, nonvolatile memory card, ROM, DVD (DVD-ROM, DVD-R), and the like.

  As another program supply method, a browser on a client computer is used to connect to an Internet home page. The computer program itself of the present invention or a compressed file including an automatic installation function can be downloaded from the homepage by downloading it to a recording medium such as a hard disk.

  It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the present invention.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and key information for decryption is downloaded from a homepage via the Internet to users who have cleared predetermined conditions. Let It is also possible to execute the encrypted program by using the downloaded key information and install the program on a computer.

  Further, the functions of the above-described embodiments are realized by the computer executing the read program. In addition, based on the instructions of the program, an OS or the like running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments can also be realized by the processing.

  Further, the program read from the recording medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Thereafter, the CPU of the function expansion board or function expansion unit performs part or all of the actual processing based on the instructions of the program, and the functions of the above-described embodiments are realized by the processing.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed as being limited thereto. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

DESCRIPTION OF SYMBOLS 100 Projection system 101 Information processing apparatus 102 Camera 103 Sensor 104 Projector 105 Reflector 106 Projection surface 107 Printer 201 CPU
202 ROM
203 RAM
204 System Bus 205 Input Controller 206 Video Controller 207 Memory Controller 208 Communication I / F (Interface) Controller 209 Input Device 210 Display 211 External Memory

Claims (15)

A projection device for projecting an image;
An information processing apparatus that is communicably connected to an imaging apparatus that images an object that is a real object existing on a projection plane projected by the projection apparatus,
An object specifying means for specifying the type of the object;
An information processing apparatus comprising: image pickup sound control means for controlling image pickup sound output in accordance with the image pickup by the image pickup apparatus in accordance with the type of the object specified by the object specifying means.   The imaging sound control means does not output the imaging sound when the object specifying means specifies that the type of the object is an indicator that is an object used for an instruction by a user. The information processing apparatus according to claim 1.   The imaging sound control means outputs the imaging sound when the object specifying means specifies that the type of the object is not an indicator that is an object used for an instruction by a user. Item 3. The information processing device according to item 1 or 2.   When the object specifying unit specifies that the type of the object is the indicator, the imaging sound control unit is more than the case where the object specifying unit specifies that the type of the object is not the indicator. The information processing apparatus according to claim 2, wherein the volume of the image pickup sound is reduced. A projection device for projecting an image;
An imaging device for imaging an object on a projection surface projected by the projection device;
A control method for an information processing apparatus connected to be communicable,
An object specifying step in which the object specifying means of the information processing device specifies the type of the object;
The image pickup sound control means of the information processing apparatus comprises an image pickup sound control step for controlling the image pickup sound output according to the image pickup of the image pickup apparatus according to the type of the object specified by the object specifying means. A method for controlling an information processing apparatus. A projection device for projecting an image;
An imaging device that images an object that is a real object existing on a projection plane projected by the projection device;
A program capable of executing a control method of an information processing apparatus connected to be communicable,
The information processing apparatus;
An object specifying means for specifying the type of the object;
A program that functions as imaging sound control means for controlling imaging sound that is output in response to imaging by the imaging device, in accordance with the type of object specified by the object identification means. A projection device for projecting an image;
An imaging device that images an object that is a real object existing on a projection plane projected by the projection device;
A projection system including an information processing apparatus,
The information processing apparatus includes:
An object specifying means for specifying the type of the object;
A projection system comprising: imaging sound control means for controlling imaging sound output in response to imaging by the imaging device in accordance with the type of object specified by the object specifying means. A projection device for projecting an image;
An imaging device that images an object that is a real object existing on a projection plane projected by the projection device;
A control method of a projection system including an information processing device,
An object specifying step in which the object specifying means of the information processing device specifies the type of the object;
The image pickup sound control means of the information processing apparatus comprises an image pickup sound control step for controlling the image pickup sound output according to the image pickup of the image pickup apparatus according to the type of the object specified by the object specifying means. A control method of a projection system characterized by the above. A projection device for projecting an image;
An imaging device that images an object that is a real object existing on a projection plane projected by the projection device;
A program capable of executing a projection system control method including an information processing apparatus,
The information processing apparatus;
An object specifying means for specifying the type of the object;
A program that functions as imaging sound control means for controlling imaging sound that is output in response to imaging by the imaging device, in accordance with the type of object specified by the object identification means. An information processing apparatus that is communicably connected to a projection apparatus that projects electronic data and an imaging apparatus that captures an image according to an imaging instruction,
Storage means for storing the picked-up sound for each process executed according to the user's operation;
Motion detection means for detecting the user's motion;
An imaging instruction transmitting means for transmitting an imaging instruction to the imaging apparatus when a user operation is detected by the operation detecting means;
In accordance with the imaging instruction transmitted by the imaging instruction transmission means, the imaging sound output means for outputting the imaging sound,
The information processing apparatus according to claim 1, wherein the imaging sound output unit acquires and outputs imaging sound corresponding to processing executed in accordance with the operation detected by the operation detection unit from the storage unit. The operation detection means detects a capture operation that is a capture operation of the user's paper document,
The information processing apparatus according to claim 10, wherein the imaging sound output unit acquires and outputs imaging sound corresponding to the capturing operation from the storage unit. The operation detection means detects an explanation operation which is an operation indicating an explanation by the user,
The information processing apparatus according to claim 10, wherein the imaging sound output unit acquires and outputs imaging sound corresponding to the explanation operation from the storage unit. The operation detection means detects an agreement operation that is an operation indicating consent by the user,
The information processing apparatus according to any one of claims 10 to 12, wherein the imaging sound output unit acquires and outputs imaging sound corresponding to the consent operation from the storage unit. An information processing apparatus including a projection apparatus that projects electronic data and an imaging apparatus that captures an image according to an imaging instruction, and a storage unit that stores imaging sound for each process executed according to a user operation A control method,
The operation detection unit of the information processing apparatus includes an operation detection step of detecting a user operation,
An imaging instruction transmitting unit of the information processing apparatus transmits an imaging instruction to the imaging apparatus when a user operation is detected in the operation detection step;
The imaging sound output means of the information processing apparatus includes an imaging sound output step that outputs an imaging sound in response to the imaging instruction transmitted by the imaging instruction transmission step.
The method for controlling an information processing apparatus, wherein the imaging sound output step acquires and outputs an imaging sound corresponding to a process executed in accordance with the operation detected in the operation detection step from the storage unit. An information processing apparatus including a projection apparatus that projects electronic data and an imaging apparatus that captures an image according to an imaging instruction, and a storage unit that stores imaging sound for each process executed according to a user operation A program capable of executing a control method,
The information processing apparatus;
Motion detection means for detecting the user's motion;
An imaging instruction transmitting means for transmitting an imaging instruction to the imaging apparatus when a user operation is detected by the operation detecting means;
In response to an imaging instruction transmitted by the imaging instruction transmission means, function as imaging sound output means for outputting imaging sound,
The image pickup sound output means executes the method of controlling the information processing apparatus, wherein the image pickup sound corresponding to the process executed in accordance with the operation detected by the action detection means is acquired from the storage means and output. Possible program.