JP2019082572A - Information processing system, information processing device, information processing method and program - Google Patents

Abstract

To enable the confirmation of data without printing by projecting the data to a medium.SOLUTION: An information processing system includes: a projection control unit for controlling projection of an image; a medium detection part for detection an area of a medium positioned within the projection range of the image; and a generation unit for generating a projection image to be projected in the projection range such that an image related to selected data corresponds to the area of the medium.SELECTED DRAWING: Figure 6

Description

  The present invention relates to an information processing system, an information processing apparatus, an information processing method, and a program.

  For electronic data such as image data and document data stored in a computer etc., use an application (for example, word processing software, spreadsheet software, presentation software, PDF viewer, etc.) for processing the electronic data. It can be displayed on a display device. The user can manipulate the displayed electronic data (page turning, editing, etc.) using the application displaying the electronic data.

  In addition, a book made of a plurality of sheets of paper is displayed on a display device capable of stereoscopic display, and two objects that perform an operation of flipping a part of the plurality of sheets of paper are turned over based on the detection result. A display device for displaying a page on a display unit has been proposed (see, for example, Patent Document 1).

  However, applications generally differ depending on the type of electronic data. Thus, the user must remember the operation of each application.

  Further, in the technology of Patent Document 1, there is a problem that it is difficult to operate because an operation is performed on a stereoscopically displayed book or paper image such as AR (Augmented Reality).

  Therefore, if electronic data is printed, page turning, writing, deletion, etc. are released from the difference in operability of each application. However, it is not only economical to print paper to read electronic data, but it also places a burden on the environment.

  The present invention has been made in view of the above-described points, and it is an object of the present invention to make it possible to confirm data without printing by projecting data onto a medium.

  Therefore, in order to solve the above problems, the information processing system includes a projection control unit that controls projection of an image, a medium detection unit that detects an area of a medium disposed in the projection range of the image, and an image regarding selected data. A generation unit that generates a projection image to be projected onto the projection range such that the image data corresponds to the area of the medium.

  By projecting the data onto the medium, the data can be confirmed without printing.

FIG. 1 is a diagram showing an example of a configuration of a data operation support system 1 in a first embodiment. It is a figure showing the example of hardware constitutions of information processor 10 in a 1st embodiment. It is a figure which shows the example of a hardware constitutions of the paper projector 20 in 1st Embodiment. It is a figure which shows the 1st example of the external appearance of the paper projector 20 in 1st Embodiment. It is a figure which shows the 2nd example of the external appearance of the paper projector 20 in 1st Embodiment. FIG. 2 is a diagram showing an example of functional configurations of an information processing apparatus 10 and a paper projection apparatus 20 in the first embodiment. It is a figure which shows the example of arrangement | positioning of the projection paper in the projection range. It is a figure which shows the structural example of UI area | region. FIG. 6 is a view showing an example of arrangement of UI areas with respect to a paper area. It is a figure which shows an example of movement and rotation of a projection paper. It is a figure which shows an example of turning of projection paper. 5 is a flowchart for describing an example of a processing procedure executed by the information processing apparatus 10 in the first embodiment. FIG. 2 is a view showing an example of the configuration of a paper area information storage unit 111; It is a flowchart for demonstrating an example of the process sequence of a drawing information corresponding | compatible process. It is a flowchart for demonstrating an example of the process sequence of a projection image data update process. It is a flowchart for demonstrating an example of the process sequence which the paper projection apparatus 20 performs in 1st Embodiment. It is a flowchart for demonstrating an example of the process sequence which the information processing apparatus 10 performs in 2nd Embodiment. It is a figure which shows the function structural example of the information processing apparatus 10 and the paper projector 20 in 3rd Embodiment. It is a flowchart for demonstrating an example of the process sequence which the information processing apparatus 10 performs in 3rd Embodiment. It is a flow chart for explaining an example of the processing procedure which paper projection device 20 performs in a 3rd embodiment.

  Hereinafter, a first embodiment will be described based on the drawings. FIG. 1 is a diagram showing an example of a configuration of a data operation support system 1 according to the first embodiment. In FIG. 1, the data operation support system 1 includes an information processing device 10 and a paper projection device 20. The information processing apparatus 10 and the paper projection apparatus 20 are connected via a cable such as a USB (Universal Serial Bus) cable, or a communication line such as a wireless or wired LAN (Local Area Network). That is, the information processing device 10 may be placed beside the paper projection device 20 (for example, on the same desk or the like), or may be placed at a distance from the paper projection device 20.

  The information processing apparatus 10 is a general-purpose computer. In the present embodiment, the information processing apparatus 10 generates an image to be projected by the paper projection apparatus 20 (hereinafter referred to as “projected image data”) and is disposed within the projection range of the paper projection apparatus 20. Execute processing according to the user's operation on paper (hereinafter referred to as "projection paper").

  The paper projection device 20 is a device that projects projection image data onto a projection range and detects an operation by a user on a projection sheet placed within the projection range.

  FIG. 2 is a diagram showing an example of a hardware configuration of the information processing apparatus 10 according to the first embodiment. The information processing apparatus 10 of FIG. 2 includes a drive unit 100, an auxiliary storage unit 102, a memory unit 103, a CPU 104, an interface unit 105, a display unit 106, an input unit 107 and the like mutually connected by a bus B.

  A program for realizing the processing in the information processing apparatus 10 is provided by a recording medium 101 such as a CD-ROM. When the recording medium 101 storing the program is set in the drive apparatus 100, the program is installed from the recording medium 101 to the auxiliary storage apparatus 102 via the drive apparatus 100. However, the installation of the program does not necessarily have to be performed from the recording medium 101, and may be downloaded from another computer via a network. The auxiliary storage device 102 stores the installed program and also stores necessary files and data.

  The memory device 103 reads out the program from the auxiliary storage device 102 and stores it when there is an instruction to start the program. The CPU 104 implements the function related to the information processing apparatus 10 in accordance with a program stored in the memory device 103. The interface device 105 is used as an interface for connecting to a network. The display device 106 displays a graphical user interface (GUI) according to a program. The input device 107 includes a keyboard, a mouse, and the like, and is used to input various operation instructions.

  FIG. 3 is a diagram showing an example of a hardware configuration of the paper projection device 20 according to the first embodiment. The paper projection device 20 of FIG. 3 includes an auxiliary storage device 201, a memory device 202, a CPU 203, an interface device 204, a projector 205, an infrared light projector 206, an infrared camera 207, and the like.

  A program for realizing the processing in the paper projection device 20 is installed in the auxiliary storage device 201. The memory device 202 reads out the program from the auxiliary storage device 201 and stores it when there is an instruction to start the program. The CPU 203 executes a function related to the paper projection device 20 in accordance with a program stored in the memory device 202. The interface device 204 is used as an interface for connecting to a network.

  The projector 205 is a device (projector) that projects image data into a predetermined range (projection range).

  The infrared light projector 206 emits infrared light to the projection range. The infrared camera 207 captures an image that visualizes the infrared radiation emitted in the projection range.

  FIG. 4 is a view showing a first example of the appearance of the paper projection device 20 in the first embodiment. In FIG. 4, (1) shows a perspective view, (2) shows a top view, (3) shows a left side view, (4) shows a front view, and (5) shows a right side view. , (6) shows a rear view.

  As shown in FIG. 4, the infrared light projector 206 is arranged to be capable of projecting infrared light onto the entire projection range A1 of the projector 205, and the infrared camera 207 is arranged to be capable of imaging the entire projection range A1. Be done. The projection sheet is disposed within the projection range A1. In FIG. 4, the projection range A1 is drawn smaller than the size of the paper projection device 20 for the sake of convenience, but it is actually possible to set a larger range as the projection range A1.

  4 shows an example in which the projector 205 and the infrared light projector 206 are exposed, for example, as shown in FIG. 5, the projector 205 and the infrared light projector 206 are provided in order to ensure an aesthetic appearance. A cover may be installed.

  FIG. 5 is a view showing a second example of the appearance of the paper projection device 20 in the first embodiment. FIG. 5 shows an example in which the cover c1 and the cover c2 are installed. The cover c1 may be any material as long as it passes through infrared light. In addition, the cover c2 has a window w1 for preventing the projection by the projector 205 from being blocked.

  FIG. 6 is a diagram showing an example of functional configurations of the information processing apparatus 10 and the paper projection apparatus 20 in the first embodiment. 6, the information processing apparatus 10 includes a selection unit 11, a conversion unit 12, a switching unit 13, a generation unit 14, an image transmission unit 15, a paper information reception unit 16, a drawing processing unit 17, and the like. These units are realized by processing that one or more programs installed in the information processing apparatus 10 are executed by the CPU 104. The information processing apparatus 10 also uses the paper area information storage unit 111. Each of the storage units can be realized using, for example, a memory device 103, an auxiliary storage device 102, or a storage device connectable to the information processing device 10 via a network.

  The selection unit 11 receives a selection of a file to be a projection target such as a PDF file (hereinafter, referred to as a “projection target file”). The conversion unit 12 generates page image data which is image data (for example, JPEG data or the like) for each page of the projection target file.

  The switching unit 13 switches a page to be a projection target (hereinafter, referred to as a “projection target page”) among the plurality of page image data.

  The generation unit 14 performs projection based on information indicating the position of the projection sheet in the projection range A1 by the sheet projection device 20, the rotation angle of the projection sheet, the size of the projection sheet, etc. (hereinafter referred to as "sheet area information"). The projection image data is generated such that the page image data of the projection target page is projected onto the area of the projection sheet (hereinafter referred to as "paper area") in the range A1.

  FIG. 7 is a view showing an arrangement example of projection sheets in the projection range. As shown in FIG. 7, the projection paper p1 is arranged to be included in the projection range A1. In FIG. 7, the area formed by the projection paper p1 is a paper area. A plurality of projection sheets p1, p2, ..., pn may be arranged in the projection range A1. In this case, the position, the rotation angle, and the like of each of the projection sheets p1 may be different.

  The paper area information may be, for example, information including information indicating the projection range A1 and information indicating the paper area in a predetermined coordinate system. The information indicating the projection range A1 may be, for example, coordinate values of each vertex of the projection range A1 in the coordinate system or a pair of coordinate points of two corners. The information indicating the paper area may be, for example, information including, in the coordinate system, coordinate values of each vertex of the paper area or two vertices of diagonals and a rotation angle.

  The projection image data is generated, for example, as an image corresponding to the entire projection range A1. In this case, in the image, the page to be projected is applied to the area corresponding to the paper area, and it is difficult for the user to recognize that the area other than the paper area (hereinafter referred to as “background area”) is projected. It may be (filled with black or dark blue) or a mode indicating a background area (pattern such as a geometric pattern, wood grain pattern of a wooden desk, etc.). Alternatively, when control of ON / OFF of the light source of the paper projection device 20 is possible, the light source may be turned OFF for the background area. Furthermore, when the projection mechanism (projection lens system or the like) of the paper projection device 20 has a function to control the projection following the object, the projection onto the paper area is not performed, and the projection onto the background area is not performed. The projection mechanism may be controlled to perform the In addition, the control of ON / OFF of the light source and the control of the projection mechanism such as the projection angle may be combined. However, the projection image data may not be image data corresponding to the entire projection range A1, but may be image data corresponding only to the projection sheet p1 and the vicinity thereof, and page image data to be projected as it is with projection image data. It may be done.

  In the drawing mode, the generation unit 14 generates projection image data so as to include a UI area for receiving a change in drawing color or the like. The drawing mode is a processing state (period) in which drawing on a projection sheet is effective. For example, it is started by a touch of a pen on a drawing sheet. The pen may be a dedicated pen or a normal pen. Hereinafter, the pen used for drawing is called "drawing pen". It is preferable that the drawing pen is a device such as a touch pen or a stylus pen that does not actually draw on an object to be drawn (the projection sheet p1 in the present embodiment). As shown in FIG. 6, etc., it may be capable of drawing on the projection paper p1.

  FIG. 8 is a diagram showing an example of the configuration of the UI area. The UI area u1 shown in FIG. 8 includes virtual (represented by an image) buttons such as the buttons b1 to b8. The button b1 is a button for receiving an instruction to return the projection target page to the previous page. The button b2 is a button for receiving a change instruction to a black ball pen or a black pen for the type of drawing pen (hereinafter, referred to as “drawing type”). The button b3 is a button for receiving a change instruction to the red ballpoint pen or the red pen for the drawing type. The button b4 is a button for receiving a change instruction to the blue ballpoint pen or the blue pen for the drawing type. The button b5 is a button for receiving a change instruction to the yellow highlighter for the drawing type. The button b6 is a button for receiving a change instruction to the eraser for the drawing type. That is, the selection states of the buttons b2 to b6 are exclusive, and the selected button is enlarged. Therefore, FIG. 8 shows a state in which the button b3 is selected. The button b7 is a button for receiving an instruction to save and end the projection. The button b8 is a button for receiving an instruction to move the projection target page one page forward. The drawing type shown in FIG. 8 is merely an example. Other drawing types may be selectable.

  The UI area u1 is disposed, for example, in the paper area as shown in FIG. FIG. 9 is a view showing an arrangement example of the UI area with respect to the paper area. In FIG. 9, rectangles shown in (1) and (2) respectively indicate paper areas. (1) shows an arrangement example of the UI area u1 when the projection paper p1 is placed in the vertical direction in the projection range A1. (2) shows an arrangement example of the UI area u1 when the projection paper p1 is placed in the lateral direction in the projection range A1. The UI area u1 may be disposed in an area other than the paper area in the projection range A1.

  Return to FIG. The image transmission unit 15 transmits the projection image data to the paper projection device 20.

  The paper information receiving unit 16 receives (acquires) paper area information from the paper projection device 20. The paper area information received by the paper information receiving unit 16 is stored in the paper area information storage unit 111. The paper information receiving unit 16 also receives, from the paper projection device 20, information (hereinafter referred to as "operation information") indicating that the projection paper p1 has been turned or the corner of the projection paper p1 has been folded. The operation information received by the paper information receiving unit 16 is notified to the switching unit 13.

  The drawing processing unit 17 receives information (hereinafter referred to as “drawing information”) indicating the drawing content of the drawing pen on the projection paper p1 from the paper projection device 20. When the drawing pen is a pen that can not actually perform drawing, the drawing information is information indicating the locus of the pen point in a state where the pen point of the drawing pen is in contact with the projection sheet p1. The drawing processing unit 17 updates the page image data to be projected based on the received drawing information. That is, the drawing processing unit 17 reflects the drawing content indicated by the drawing information on the page image data. At this time, the reflection of the drawing content is performed according to the selected drawing type.

  On the other hand, the paper projection device 20 includes an image reception unit 21, a projection control unit 22, a paper detection unit 23, a drawing detection unit 24, and the like. These units are realized by processing that one or more programs installed in the paper projection device 20 cause the CPU 203 of the paper projection device 20 to execute.

  The image reception unit 21 receives projection image data from the information processing apparatus 10. The projection control unit 22 controls the projection of the projection image data received by the image reception unit 21 to the projection range A1 by the projector 205.

  The paper detection unit 23 detects a paper area such as the position, the rotation angle, and the size for each projection paper p1 in the projection range A1 based on the contrast of the infrared light indicated by the image input from the infrared camera 207. For example, the paper detection unit 23 detects, as a paper region, a region where the pixel value in the image is equal to or greater than a threshold. The paper detection unit 23 also detects a change in the paper area. For example, the paper detection unit 23 detects movement of the projection sheet p1, rotation of the projection sheet p1, removal or addition of the projection sheet p1, and the like. The paper detection unit 23 transmits paper area information indicating the paper area to the information processing apparatus 10 when the paper area is first detected after activation of the paper projection device 20 or when a change in the paper area is detected.

  FIG. 10 is a view showing an example of movement and rotation of a projection sheet. In FIG. 10, (1) shows an example of the movement of the projection sheet p1 within the projection range A1. (2) shows an example of rotation of the projection sheet p1. As a result of the rotation, as shown in (3), the projection paper p1 may have an inclination with respect to the projection range A1. For example, this inclination is the rotation angle of the projection sheet p1.

  When a plurality of projection sheets p1 are disposed in the projection range A1, the sheet detection unit 23 detects changes in the sheet area and the sheet areas of the respective projection sheets p1. Therefore, for example, by arranging the projection paper p1 in front of each attendee in a meeting, a meeting or the like, page image data can be projected in front of each attendee. In this case, the paper projection device 20 may be arranged at a high place in order to enlarge the projection range A1.

  Return to FIG. The paper detection unit 23 also detects an operation that meets a predetermined condition from the user's operation (for example, turning, folding of a corner, etc.) on the projection paper p1.

  FIG. 11 is a view showing an example of turning of a projection sheet. As shown in FIG. 11, the turning of the projection sheet p1 refers to, for example, an operation of lifting the side of the projection sheet p1 so as to scan the right or left side and then returning the lifted side. Therefore, resentment is different from inside out. However, turning may be substituted by turning over.

  For example, when the user manipulates the projection sheet p1, the paper area returns to the original size and shape after the size and shape of the paper area temporarily change. The paper detection unit 23 detects the turning of the projection paper p1 based on the detection of the change in the paper area that meets such conditions. The paper detection unit 23 transmits, to the information processing apparatus 10, operation information indicating the content of the operation on the projection paper p1. The turning of the projection sheet p1 corresponds to a change instruction of the projection target page.

  In addition, when the user performs an operation of temporarily folding any corner of the projection sheet p1, the corner portion of one of the four corners of the paper area changes to a scraped shape, and then returns to the original shape. The paper detection unit 23 detects the folding of the corner of the projection paper p1 based on the detection of such a phenomenon. When the folding of the corner of the projection sheet p1 is detected, the information processing apparatus 10 determines that there is a bookmark insertion instruction (or tag addition instruction) for the projection target page.

  The drawing detection unit 24 detects drawing (a locus of a pen point) by the drawing pen on the projection paper p1 included in the projection range A1.

  Various methods can be used as a drawing detection method. For example, a method of detecting the locus of the pen tip with a drawing pen provided with a small camera that picks up a pattern printed on the projection paper p1 may be employed. Also, a method of detecting the locus of the pen tip on the projection sheet p1 by imaging with a camera different from the drawing pen may be employed. Further, a method may be employed in which the drawing content is detected by a projection sheet p1 (panel) that detects drawing by a pressure-sensitive method or an electrostatic method. Further, a method is employed in which the infrared light projector 206 projects infrared light in a grid shape to the projection range A1, and the infrared camera 207 detects drawing by a method of detecting the grid passed by the pen tip of the drawing pen. It is also good. In addition, when the drawing pen actually draws on the projection paper p1, a method of detecting the actually drawn content as an image may be used. The drawing detection unit 24 transmits, to the information processing apparatus 10, drawing information including a coordinate group indicating the detected drawing content (locus of the pen tip). Alternatively, image data in which drawing content is reflected on page image data that is a part of projection target image data or projection target image data may be transmitted to the information processing apparatus 10 as drawing information.

  Hereinafter, processing procedures executed by the information processing apparatus 10 and the paper projection apparatus 20 will be described. FIG. 12 is a flowchart for explaining an example of the processing procedure executed by the information processing apparatus 10 in the first embodiment.

  For example, when the paper information receiving unit 16 first detects a paper area in the paper projection device 20, when a change in the paper area is detected due to movement or rotation of the projection paper p1, or the paper area increases or decreases. When the paper area information transmitted from the paper projection device 20 is received (when the projection paper p1 increases or decreases) (Yes in S101), the paper information receiving unit 16 receives the received paper area information as the paper area information storage unit 111. (S102).

  FIG. 13 is a view showing a configuration example of the paper area information storage unit 111. As shown in FIG. As shown in FIG. 13, each record of the paper area information storage unit 111 includes items such as “projected file name”, “coordinate information”, “rotation angle”, “projected page”, and “drawing mode”. Including. “Projection target file name” is a file name of a projection target file. "Coordinate information" is information indicating the position and range of the paper area. The “coordinate information” is, for example, coordinate values of the apexes of the diagonal of the paper area. The “rotation angle” is information indicating the rotation angle of the paper area. The “projected page” is the page number of the page currently being projected in the projected file. The drawing mode is information indicating whether the drawing mode is in progress (TRUE) or not (FALSE). Among these items, "coordinate information" and "rotation angle" are included in the paper area information. Therefore, the paper area information storage unit 111 is configured to be able to store paper area information in association with the projection target age, the drawing mode, and the like. The paper area information storage unit 111 stores a record for each of the paper area information.

  For example, when the paper area is detected first, when the plurality of projection papers p1p2, ..., pn are arranged within the projection range A1, the paper area information of each of the projection papers p1p2, ..., pn Is received and stored in the paper area information storage unit 111.

  In addition, when the paper area of any of the projection sheets p1 changes (moves or rotates), the paper area information before changing (before moving or rotating) and the paper area information after changing (after moving or rotating) It is received. In this case, the sheet information receiving unit 16 updates the sheet area information before moving or rotating among the sheet area information stored in the sheet area information storage unit 111 to the sheet area information after moving or rotating. .

  Further, when any projection sheet p1 is removed from the projection range A1, in addition to the paper area information in the state before the removal of the projection sheet p1, the information indicating that the projection sheet p1 has been removed is received . In this case, the paper information receiving unit 16 stores the paper area information stored in the paper area information storage unit 111 and the information associated with the paper area information as described later into the paper area information storage unit 111. Remove from

  When a new projection sheet p1 is added to the projection range A1, the paper area information after the projection sheet p1 is arranged is received. In this case, the paper information receiving unit 16 stores the paper area information in the paper area information storage unit 111.

  When synchronizing the projection target page, the drawing mode, and the like for all the projection sheets p1, it is not necessary to distinguish the respective projection sheets p1, so in step S101, they relate to the latest paper area of each of all the projection sheets p1. Paper area information may be received each time. In this case, in the paper area information storage unit 111, all the paper area information may be rewritten every time.

  Alternatively, when the selection unit 11 receives the selection of the projection target file (Yes in S103), the conversion unit 12 reads the projection target file into the memory device 103, and the projection target file in the memory device 103 (hereinafter, “projection target file Page image data is generated for each page of the file m1)) (S104). For example, if the projection target file m1 is a PDF file including five pages, five JPEG data are generated as page image data. The generated page image data is stored, for example, in the auxiliary storage device 102. Subsequently, the switching unit 13 selects the first page in the projection target file m1 as the projection target page (S105). Specifically, the switching unit 13 stores information (for example, “1”) indicating the page to be projected in the paper area information storage unit 111 in association with the respective paper area information stored in the paper area information storage unit 111. .

  Alternatively, when the sheet information receiving unit 16 receives the operation information from the sheet projection apparatus 20 (Yes in S106), the sheet information receiving unit 16 displays the operation information (hereinafter referred to as "target operation information") on the projection sheet p1. It is determined whether it is information indicating turning or information indicating folding of a corner of the projection sheet p1 (S107).

  If the target operation information is information indicating turning of the projection sheet p1 (Yes in S107), the switching unit 13 changes the projection target page according to the operation content indicated by the target operation information (S108). In this case, the target operation information includes the paper area information of the projection sheet p1 which is the target of the turning as identification information of the projection sheet p1. Therefore, for example, when the target operation information indicates an operation of lifting the right side of the projection sheet p1 and turning it to the left, the switching unit 13 displays the projection target page associated with the target operation information in the paper area information storage unit 111. Add 1 (ie, the next page is the projection target page). On the other hand, when the target operation information indicates an operation of lifting the left side of the projection sheet p1 and turning it to the right, the switching unit 13 selects one from the projection target page associated with the target operation information in the paper area information storage unit 111. Subtract (ie, make the previous page the projection target page). However, when the projection target page is already the final page, the change to the next page is not performed, and when the projection target page is the first page, the change to the previous page is not performed. However, if the page to be projected is already the final page and switching to the next page is requested, the first page may be set as the page to be projected, or the page to be projected is the first page. In the case where the switch to the previous page is requested, the final page may be set as the projection target page. The right and left sides of the projection sheet p1 are based on the direction of the page image projected on the projection sheet p1. Specifically, with respect to a state where the vertical direction of the page image matches the user (that is, the upper direction of the page image is away from the user and the lower direction is closer to the user), The right and left sides are decided. However, the direction of the turning may not be distinguished. For example, even if the projection sheet p1 is turned in any direction, the next page or the previous page may be set as the projection target page.

  Alternatively, the switching unit 13 is associated with the target operation information in the paper area information storage unit 111 regardless of the direction of the ringing included in the target operation information when the operation information indicating the ringing is received for the first time. One may be added to the projection target page. In this case, the switching unit 13 stores an operation related to the operation information as an operation to advance the page, and stores an operation reverse to the operation as an operation to return the page. For example, when the operation information received for the first time indicates an operation of lifting the right side of the projection sheet p1 and turning it to the left, the operation is stored as an operation for advancing the page, and an operation such as lifting the left side and turning to the right (turn over). Is stored as an operation to return the page. The switching unit 13 switches pages based on the stored content.

  Alternatively, for example, when the page image data is a horizontally long image and the page image data faces the user in a state where the projection sheet p1 is placed in the horizontal direction, the upper side or the lower side of the projection sheet p1 is turned Thus, turning of the projection sheet p1 may be detected. In this case, the switching unit 13 may advance the projection target page when the lower side is turned and return the projection target page when the upper side is turned.

  In the present embodiment, in the case where a plurality of previous projection sheets p1 are arranged within the projection range A1, an example in which the respective projection contents are different (independent) is described. When the projection paper p1 is arranged within the projection range A1 and when it is desired to project the same projection target page on all the projection paper p1, that is, when it is desired to project synchronously, the projection target page is Rather, only one need be stored. Alternatively, in step S108, all projection target pages stored for each sheet area information in the sheet area information storage unit 111 may be updated.

  On the other hand, when the target operation information is information indicating that the corner of the projection sheet p1 has been folded (No in S107), the paper information receiving unit 16 adds (adds) bookmark information to the projection target file ( S109). That is, the paper information receiving unit 16 refers to the projection target page stored in the paper area information storage unit 111 in association with the paper area information included in the target operation information, and among the pages of the projection target file, Bookmark information is added (added) to the page corresponding to the projection target page (S109). That is, as described later, in this case, the target operation information includes sheet area information of the projection sheet p1 whose corner is folded.

  Alternatively, when the drawing processing unit 17 receives drawing information (hereinafter referred to as “target drawing information”) from the paper projection device 20 (Yes in S110), the drawing processing unit 17 executes a drawing information correspondence process (S111). ). In the drawing information handling process, a process according to the target drawing information is executed. Details of the drawing information handling process will be described later.

  Subsequent to steps S102, S105, S107 and S111, the information processing apparatus 10 executes a projection image data update process (S112). After execution of step S112, step S101 and subsequent steps are repeated. The details of the projection image data update process will be described later.

  Subsequently, the details of step S111 will be described. FIG. 14 is a flowchart for explaining an example of the processing procedure of the drawing information correspondence process.

  In step S201, the drawing processing unit 17 determines whether the drawing mode corresponding to the paper area information included in the target drawing information (that is, the paper area information corresponding to the projection paper p1 set as the drawing target) is TRUE. Determine if The drawing mode is a parameter indicating whether drawing on the projection sheet p 1 is in a valid state, and is stored in the sheet area information storage unit 111 in association with each sheet area information. As for the drawing mode, "TRUE" indicates that drawing is in a valid state, and "FALSE" indicates that drawing is in an invalid state. For example, the drawing mode can be set to “TRUE” by touching the drawing pen on the projection paper p1. When the drawing modes of all the projection sheets p1, p2,..., Pn are synchronized, only one drawing mode may be stored instead of each sheet area information.

  When the drawing mode stored in the paper area information storage unit 111 in association with the paper area information included in the target drawing information is “FALSE” (No in S201), the drawing processing unit 17 determines the paper area The drawing mode stored in the paper area information storage unit 111 in association with the information is set to "TRUE" (S202). That is, the fact that the drawing information has been received for the projection paper p1 for which the drawing mode is “FALSE” means that the drawing pen has been touched on the projection paper p1. Therefore, in this case, the drawing mode corresponding to the projection sheet p1 is updated to "TRUE".

  On the other hand, when the drawing mode stored in the paper area information storage unit 111 in association with the paper area information included in the target drawing information is “TRUE” (Yes in S201), the drawing processing unit 17 performs drawing It is determined whether the position is included in the UI area u1 (S203). Whether or not the drawing position is included in the UI area u1 is determined by coordinate information indicating the drawing position (contact position of the drawing pen) included in the target drawing information and coordinate information indicating the range of the UI area u1. It is possible by comparison. Since the UI area u1 is arranged at a predetermined position of the paper area, the coordinate information indicating the range of the UI area u1 can be specified based on the paper area information included in the target drawing information.

  When the drawing position is included in the UI area u1 (Yes in S203), the drawing processing unit 17 executes a process according to the button operated on in the UI area u1 (FIG. 8) (S204).

  For example, when the drawing position (that is, the touch position of the drawing pen) is included in the range of the button b1, the drawing processing unit 17 associates the paper area information included in the target drawing information with the paper area information storage unit 1 is subtracted from the projection target page stored in 111, and the process of step S112 (FIG. 15) is called. However, when the projection target page is “1”, nothing may be executed and the final page may be set as the projection target page. Similarly, when the drawing position is included in the range of the button b 8, the drawing processing unit 17 associates the paper area information included in the target drawing information with the projection target page stored in the paper area information storage unit 111. Is added to 1 and the process of step S112 (FIG. 15) is called. However, if the projection target page is the final page of the projection target file, nothing may be executed and the first page may be set as the projection target page.

  Further, when the drawing position is in the range of any of the buttons b2 to b6, the drawing type corresponding to the button relating to the drawing position is related to the paper area information included in the target drawing information, and the paper area information It is stored in the storage unit 111.

  Further, when the drawing position is within the range of the button b7, the drawing processing unit 17 generates a file having the same data format as the projection target file, for example, based on each page image data of the projection target file m1. The file is stored, for example, in the auxiliary storage device 102. At this time, the projection target file may be overwritten and saved, or may be stored as a file different from the projection target file. Further, the drawing processing unit 17 deletes (clears) the projection target file m1 and the page image data and the contents of the paper area information storage unit 111 after the file is stored. As a result, the projection of the projection target file m1 ends.

  On the other hand, when the drawing position is not included in the UI area u1 (No in S203), the drawing processing unit 17 is stored in the paper area information storage unit 111 in association with the paper area information included in the target drawing information. According to the selected drawing type, based on the drawing content included in the target drawing information, the page image data of the projection target page stored in the paper area information storage unit 111 in association with the paper area information is updated. (S205). Specifically, the drawing processing unit 17 executes drawing on the page image data according to the locus (coordinate group) included in the target drawing information according to the pen type of the drawing type and the color of the pen. . In addition, when the eraser is selected as the drawing type, the drawing processing unit 17 deletes the drawing content of the portion corresponding to the locus indicated by the target drawing information in the page image data.

  Subsequently, the details of step S112 will be described. FIG. 15 is a flowchart for explaining an example of a processing procedure of projection image data update processing.

  In step S301, the generation unit 14 determines whether page image data has been generated. That is, it is determined whether the projection target file has already been selected and steps S104 and S105 of FIG. 12 have been executed. If the page image data has not been generated (No in S301), the subsequent processing is not executed. If the paper area is detected by the paper projection apparatus 20 before the file to be projected is selected and step S112 is called in response to the reception of the paper area information indicating the paper area, the determination in step S301 is No. .

  When page image data has been generated (Yes in S301), the generation unit 14 determines whether one or more pieces of paper area information are stored in the paper area information storage unit 111 (S302). When one or more pieces of paper area information are stored in the paper area information storage unit 111 (Yes in S302), the generation unit 14 projects the page image data of the projection target page in the paper area based on the respective paper area information. Image data is generated (S303). That is, in the projection image data in which the projection range A is the entire area, the page image data of the projection target page associated with each sheet area information is superimposed on the sheet area indicated by each sheet area information. At this time, each page image data is arranged to be enlarged or reduced and rotated according to the size and the rotation angle indicated by each sheet area information. Therefore, the page image data for the paper area corresponding to the A4 projected paper p1 is enlarged or reduced to fit in the A4 size, and the page image data for the paper area corresponding to the A3 projected paper p1 is A3. It is scaled up or down to fit the size. The above description does not intend that the size of the projection sheet p1 is limited to A3 or A4, and any size may be selected as the size of the projection sheet p1.

  Subsequently, the generation unit 14 determines whether or not the paper area information (projection paper p1) associated with the drawing mode “TRUE” is stored in the paper area information storage unit 111 (S305). If there is no corresponding paper area information (S305: No), the process proceeds to step S307. When the corresponding paper area information exists (Yes in S305), the generation unit 14 combines (superimposes) an image indicating the UI area u1 at a predetermined position of the paper area related to the paper area information in the projection image data ( S306). That is, projection image data including the UI area u1 in the paper area is generated.

  On the other hand, when no paper area information is stored in the paper area information storage unit 111 (No in S302), the generation unit 14 generates projection image data whose entire surface is a background area (S304). The paper area information is deleted from the paper area information storage unit 111 in response to the removal of the projection paper p1. Therefore, when all the projection sheets p1 are removed, it appears to the user that the page image data has been hidden.

  In the case of No at step S305, following step S306 or step S304, the image transmission unit 15 transmits the generated projection image data to the paper projection device 20 (S307).

  Subsequently, a processing procedure performed by the paper projection device 20 will be described. FIG. 16 is a flow chart for explaining an example of the processing procedure executed by the paper projection device 20 in the first embodiment.

  For example, if the paper detection unit 23 detects a new paper area in the projection range A1 or if it detects a change in any of the paper areas already detected (Yes in S401), the paper detection unit 23 The paper area information indicating the paper area newly detected in the projection range A1 or the paper area information corresponding to the changed paper area is transmitted to the information processing apparatus 10 (S402). When a plurality of paper areas are detected, paper area information of each of the plurality of paper areas is transmitted. In addition, when movement or rotation of any paper area is detected, the paper area information before movement or rotation of the paper area and the paper area information after movement or rotation are transmitted to the information processing apparatus 10 Ru. In addition, when any projection sheet p1 is removed from the projection range A1, in addition to the sheet area information indicating the sheet area before the removal of the projection sheet p1, the projection sheet p1 corresponding to the sheet area information is removed The information indicating that is transmitted to the information processing apparatus 10.

  When synchronizing the projection target page, the drawing mode, and the like for all the projection sheets p1, it is not necessary to distinguish the respective projection sheets p1. Therefore, in step S401, the latest sheet area of all the projection sheets p1 Paper area information may be sent.

  Alternatively, when the image reception unit 21 receives the projection image data transmitted from the information processing apparatus 10 (Yes in S403), the projection control unit 22 executes the projection of the projection image data on the projection range A1 to the projector 205. (S404). Here, page image data is applied to each paper area in the projection image data. Therefore, page image data is projected on the surface of the projection sheet p1 disposed in the projection range A1. When the projection sheet p1 is moved or rotated, the page image data is superimposed on the projection image data in accordance with the sheet area after movement or rotation. Therefore, when viewed from the user, the page image data appears to move or rotate following the projection sheet p1. Therefore, for example, the user brings the page image data to himself or another person (the next person, the person in front, the person in the publication, etc.) by bringing the projection sheet p1 close to his own direction or rotating the projection sheet p1. It can be made easy to see.

  Alternatively, when the paper detection unit 23 detects that any of the projection paper p1 has been turned (Yes in S405), the paper detection unit 23 detects the paper area information of the turned projection paper p1 and the direction of the curl. Operation information including the information to be shown is transmitted to the information processing apparatus 10 (S406). The information indicating the direction of turning is, for example, information indicating whether the right side of the projection sheet p1 is lifted and turned to the left or the left side of the projection sheet p1 is lifted and turned to the right.

  Alternatively, when the paper detection unit 23 detects that the corner of any of the projection sheets p1 has been temporarily folded (Yes in S407), the paper detection unit 23 determines the sheet area information of the projection sheet p1 and the corner. Operation information including information indicating that the key has been broken is transmitted to the information processing apparatus 10 (S408).

  Alternatively, when the drawing detection unit 24 detects drawing by the drawing pen on the sheet surface of one of the projection sheets p1 (Yes in S409), the drawing detection unit 24 generates sheet area information of the projection sheet p1 to be drawn and drawing. The drawing information including the content (the locus of the pen tip of the drawing pen) is transmitted to the information processing apparatus 10 (S410). For example, when the drawing pen touches the paper surface, information including the paper area information of the projection paper p1 related to the paper surface and the coordinates of the touched position is the drawing information. Further, when the pen point of the drawing pen moves on the paper surface, the information including the paper area information of the projection paper p1 relating to the paper surface and the coordinate group of the moved locus becomes the drawing information.

  The information processing apparatus 10 performs Step S112 after Step S105 in FIG. 12 is performed (that is, Step S112 is performed immediately after selection of the projection target file). In step S303, projection image data of each page of the projection target file m1 may be transmitted, and in step S304, projection image data of each page may be transmitted. In this case, the paper projecting device 20 projects the projection image data of the first page after projecting the projection image data of all the pages sequentially (continuously) in step S304 of FIG. Good. By doing so, when the projection starts, the user is made to grasp the sense of volume (total number of pages) of the entire projection target file that the user can feel by the thickness of the printed matter when the projection target file is printed. it can.

  As described above, according to the first embodiment, since the image of the page of the electronic data (projection target file) of the operation target is projected onto the projection sheet p1, the user can set the page via the projection sheet p1. You can browse the Further, the user can operate the electronic data (for example, add a page turning or a bookmark, etc.) by operating the projection sheet p1. Further, the user can edit the electronic data by drawing on the projection paper p1 with the drawing pen. Therefore, according to the present embodiment, it is possible to confirm the data without printing by projecting the data onto the medium.

  Next, a second embodiment will be described. In the second embodiment, points different from the first embodiment will be described. Therefore, points that are not particularly mentioned may be the same as in the first embodiment.

  In the second embodiment, an example in which page image data of different files are projected for each of a plurality of projection sheets p1 in the projection range A1 (that is, the relationship between the projection target file and the projection sheet p1 is 1 to 1) Is an example). In the second embodiment, the processing procedure of FIG. 17 is executed instead of FIG.

  FIG. 17 is a flowchart for explaining an example of the processing procedure executed by the information processing apparatus 10 in the second embodiment. In FIG. 17, the same steps as those in FIG. 12 have the same step numbers, and the description thereof will be omitted.

  In the second embodiment, when the determination in step S103 is YES (that is, when the selection of the projection target file is accepted), steps S121 and S122 are executed before step S104.

  In step S121, the converting unit 12 calculates the total number of the existing projection target file m1 and the newly selected projection target file (hereinafter referred to as the “number of projection target files”) as the paper area information storage unit 111. It is determined whether it is equal to or less than the number of sheet area information stored in (ie, the number of sheets of projection sheets p1).

  If the number of projection target files is equal to or less than the number of paper area information (Yes in S121), the conversion unit 12 assigns the newly selected projection target file to unused paper area information (S122). Unused paper area information means paper area information to which none of the existing projection target file m1 is allocated (that is, a paper area of the projection paper p1 which is not a projection destination of any of the existing projection target file m1 Information). The assignment of the projection target file to the paper area information may be performed, for example, by associating the file path name of the projection target file with the paper area information. Subsequently, step S104 and subsequent steps are performed.

  Further, in the second embodiment, in step S303 of FIG. 15, the generation unit 14 projects, for each of the paper area information stored in the paper area information storage unit 111, the projection target associated with the paper area information. The page image data corresponding to the file and the projection target page is superimposed on the projection target image data so as to be applied to the paper area indicated by the paper area information in the projection target image data. As a result, projection image data is generated in which page image data of different projection target files are projected onto the respective projection sheets p1.

  Next, a third embodiment will be described. In the third embodiment, points different from the first or second embodiment will be described. Therefore, points that are not particularly mentioned may be the same as in the first or second embodiment.

  FIG. 18 is a diagram showing an example of functional configurations of the information processing device 10 and the paper projection device 20 in the third embodiment. In FIG. 18, the same parts as those in FIG. In FIG. 18, the paper projection device 20 further includes a gesture detection unit 25. The gesture detection unit 25 is realized by processing that one or more programs installed in the paper projection device 20 cause the CPU 203 of the paper projection device 20 to execute.

  When the gesture detection unit 25 detects a predetermined gesture (motion such as gesture or hand gesture) by the user, the gesture detection unit 25 transmits information (hereinafter, referred to as “gesture information”) indicating the content of the gesture to the information processing apparatus 10. As a method of detecting a gesture, it is possible to adopt motion detection by a general motion sensor or image recognition.

  On the other hand, the information processing apparatus 10 further includes a gesture processing unit 18. The gesture processing unit 18 is realized by processing that the CPU 104 executes one or more programs installed in the information processing apparatus 10.

  When the gesture processing unit 18 receives the gesture information, the gesture processing unit 18 executes a process according to the gesture information.

  FIG. 19 is a flowchart for explaining an example of the processing procedure executed by the information processing apparatus 10 in the third embodiment. In FIG. 19, the same steps as in FIG. 12 have the same step numbers, and the description thereof will be omitted. In FIG. 19, steps S131 and S132 are added.

  When the gesture processing unit 18 receives the gesture information transmitted from the paper projection device 20 (Yes in S131), the gesture processing unit 18 executes a process according to the gesture indicated by the gesture information (S132).

  For example, when any projection sheet p1 is torn (hereinafter referred to as "gesture A"), gesture information including sheet area information on the projection sheet p1 and information indicating that the projection sheet p1 is torn. Is received. In this case, the gesture processing unit 18 deletes the page image data associated with the paper area information in step S132. Therefore, when storage is instructed after that, the projection target file corresponding to the page image data is stored in a state where the page related to the page image data is removed. The projection sheet p1 is removed from the projection range A1 by tearing the projection sheet p1. Therefore, in this case, after the paper area information indicating that the projection paper p1 has been removed is received (Yes in S101) and the paper area information is deleted from the paper area information storage unit 111, step S112 is performed. Be done. Therefore, projection image data in which page image data is not projected onto the projection sheet p1 is generated.

  When the projection sheet p1 facing a certain user moves on the projection range A1 and is handed over to the opposite user (hereinafter referred to as "gesture B"), the sheet area information of the projection sheet p1 and Gesture information including information indicating that the projection paper p1 has been handed over is received. In this case, the gesture processing unit 18 changes the rotation angle of the paper area information by 180 degrees in step S132. Therefore, in this case, in step S112, which is subsequently executed, projection image data is generated in which the page image data for the projection paper p1 is inverted so as to face the opposite user. In addition, since the position information of the paper area information stored in the paper area information storage unit 111 for the projection paper p1 is already updated with the movement of the projection paper p1 (S102). In step S132, the position information of the paper area information may not be updated.

  In addition, when one of the projection sheets p1 is lifted (hereinafter referred to as “gesture C”), the paper area information before the projection sheet p1 is lifted and information indicating that the projection sheet p1 is lifted. Gesture information including is received. In this case, the gesture processing unit 18 adds flag information indicating that it has been lifted to the paper area information stored in the paper area information storage unit 111 in step S132. As a result, in step S303 of FIG. 15, the generation unit 14 enlarges the page image data corresponding to the paper area information to which the flag information is attached so that the page image data extends beyond the paper area indicated by the paper area information. Superimpose on the data. At this time, the degree of enlargement may be changed according to the lifted height or the distance between the lifted projection paper p1 and the face (or eyes) of the user who has lifted. In this case, the height or the distance may be included in the gesture information.

  Also, if one of the projection sheets p1 is folded in half in the long side direction and then opened (returned to the original position) (for example, the sheet in the lateral direction A3 is folded so as to be A4 size) When opened (hereinafter referred to as “gesture D”), the original paper area information (for example, in the state of A3) of the projection paper p1 and that the projection paper p1 is temporarily folded in half Gesture information is received that includes the indicated information. In this case, the gesture processing unit 18 adds information indicating that the sheet area information stored in the sheet area information storage unit 111 is temporarily folded in half in step S132. If there is paper area information to which the flag information is attached in step S303 of FIG. 15, the generation unit 14 adds the paper area information to the left half of the paper area indicated by the paper area information in the projection image data. The page image data of the associated projection target page is superimposed, and the page image data of the next page of the projection target page is superimposed on the right half. As a result, on the projection paper p1 corresponding to the paper area information, two pages are aggregated and projected.

  In addition, when a plurality of projection sheets p1 on which different page image data are projected are bundled (for example, after the projection sheets p1 are overlapped, the projection sheets p1 are lifted in a substantially vertical direction) When aligned on a desktop or the like where a ton and projection range A1 are formed (hereinafter referred to as "gesture E"), the overlapping order of the respective paper area information of the plurality of projection sheets p1 and the plurality of projection sheets Gesture information including information indicating that p1 has been bundled is received. In this case, in step S132, the gesture processing unit 18 newly arranges the page image data corresponding to each of the plurality of pieces of paper area information included in the gesture information in the overlapping order page included in the gesture information. A file is generated, and the file is stored, for example, in the auxiliary storage device 102.

  FIG. 20 is a flowchart for explaining an example of the processing procedure executed by the paper projection device 20 in the third embodiment. In FIG. 20, the same steps as in FIG. 16 have the same step numbers, and the description thereof will be omitted. In FIG. 20, steps S411 and S412 are added.

  When the gesture detection unit 25 detects a gesture (action of the user on the projection paper p1) satisfying any of the gestures A to E (Yes in S411), the paper area information of the projection paper p1 which is the target of the gesture The gesture information including the information indicating the content of the gesture (the types of gesture A to gesture E) is transmitted to the information processing apparatus 10 (S412).

  The third embodiment may be combined with the second embodiment.

  In each of the above-described embodiments, bending or bending of the projection sheet p1 may also be detected based on detection of a gesture using a motion sensor or the like instead of the infrared light projector 206 and the infrared camera 207.

  The paper area in the projection range A1 may also be detected using a normal digital camera instead of the infrared light projector 206 and the infrared camera 207.

  In each of the above-described embodiments, the information processing apparatus 10 may be connected to the paper projection apparatus 20 via the Internet or the like. For example, the information processing apparatus 10 may be a computer configuring a cloud system. In this case, the selection of the projection target file may be performed using a browser or the like at a terminal connected to the information processing apparatus 10 via the Internet or the like.

  In addition, the paper projection device 20 may have all of the functions of the information processing device 10 in FIG. 6 or 18. For example, a program that causes the information processing apparatus 10 to function as each part of the information processing apparatus 10 in FIG. 6 or 18 may be installed in the paper projection apparatus 20, or the information processing apparatus 10 is included in the housing of the paper projection apparatus 20. It may be possible to

  Also, the paper projection device 20 does not have the infrared light projector 206 and the infrared camera 207, and another device having the infrared light projector 206 and the infrared camera 207 is connected to the paper projection device 20 via a communication line. May be In this case, the paper projection device 20 has the image receiving unit 21 and the projection control unit 22 in FIG. 6 or 18, and the other device corresponds to the paper detection unit 23, the drawing detection unit 24 in FIG. You may have the gesture detection part 25. FIG.

  In each of the above-described embodiments, the sheet medium to which the image is to be projected is paper. However, the medium is a medium other than paper such as "sheet" or "film" and has an area on which an image can be projected. Each of the above embodiments may be applied. At this time, as the medium, a sheet-like medium (for example, a medium which can be placed flatly on a desk or the like) is suitable.

  In addition, each function of the embodiment described above can be realized by one or more processing circuits. Here, the “processing circuit” in the present specification is designed to execute a processor programmed to execute each function by software like a processor implemented by an electronic circuit, and to execute each function described above. It includes devices such as application specific integrated circuits (ASICs), digital signal processors (DSPs), field programmable gate arrays (FPGAs), and conventional circuit modules.

  In each of the above embodiments, data operation support system 1 is an example of an information processing system. The drawing processing unit 17 is an example of an updating unit. The gesture detection unit 25 is an example of a motion detection unit. The gesture processing unit 18 is an example of a process execution unit. The paper information receiving unit 16 is an example of an acquisition unit. The paper detection unit 23 is an example of a medium detection unit.

  Although the embodiments of the present invention have been described above in detail, the present invention is not limited to such specific embodiments, and various modifications may be made within the scope of the present invention as set forth in the claims. Modifications and changes are possible.

Reference Signs List 10 information processing apparatus 11 selection unit 12 conversion unit 13 switching unit 14 generation unit 15 image transmission unit 16 sheet information reception unit 17 drawing processing unit 18 gesture processing unit 20 paper projection device 21 image reception unit 22 projection control unit 23 paper detection unit 24 Drawing detection unit 25 Gesture detection unit 100 Drive device 101 Recording medium 102 Auxiliary storage device 103 Memory device 104 CPU
105 interface device 111 paper area information storage unit 201 auxiliary storage device 202 memory device 203 CPU
204 interface device 205 projector 206 infrared projector 207 infrared camera B bus

JP, 2016-76234, A

Claims (11)

A projection control unit that controls projection of an image;
A medium detection unit that detects an area of the medium disposed in the projection range of the image;
A generation unit that generates a projection image to be projected onto the projection range such that an image related to selected data corresponds to a region of the medium;
An information processing system characterized by having. The generation unit generates the projection image so as to correspond to the area of the medium after the change when the area of the medium detected by the medium detection unit changes.
The information processing system according to claim 1, characterized in that: The generation unit changes an image to be made to correspond to the area of the medium when a change in the area of the medium detected by the medium detection unit matches a predetermined condition.
The information processing system according to claim 1 or 2, characterized in that: The generation unit is configured to, of the images of the plurality of pages included in the data, when a change in the area of the medium detected by the medium detection unit matches a condition indicating that the medium has been shaken. , Changing the image of the page corresponding to the area of the medium,
The information processing system according to claim 3, characterized in that: A drawing detection unit that detects drawing on the medium;
An updating unit that updates the data based on the content of the drawing;
The information processing system according to any one of claims 1 to 4, further comprising: An operation detection unit that detects an operation of a user targeting the medium;
A process execution unit that executes a process according to the operation detected by the operation detection unit on the data;
The information processing system according to any one of claims 1 to 5, characterized in that The medium is a sheet medium,
The information processing system according to any one of claims 1 to 6, characterized in that: The sheet medium is paper,
The information processing system according to claim 7, characterized in that. An acquisition unit that acquires information indicating an area of the medium disposed in a projection range of the image;
A generation unit that generates a projection image to be projected onto the projection range such that an image related to selected data corresponds to a region of the medium;
An information processing apparatus comprising: An acquisition procedure for acquiring information indicating an area of the medium disposed in a projection range of the image;
A generation procedure for generating a projection image to be projected onto the projection range such that an image related to selected data corresponds to an area of the medium;
An information processing method characterized in that the computer executes. An acquisition procedure for acquiring information indicating an area of the medium disposed in a projection range of the image;
A generation procedure for generating a projection image to be projected onto the projection range such that an image related to selected data corresponds to an area of the medium;
A program characterized by causing a computer to execute.

Images