JP6149905B2 - Information processing apparatus and program - Google Patents

Description

  The present invention relates to an information processing apparatus and a program.

  In recent years, in meetings and presentations, there are many opportunities for explanation while displaying materials on a PC (Personal Computer) or a tablet terminal. Participants take notes on notes and points they notice while browsing the displayed materials. The material may be distributed as an electronic file, but there is a risk that the correspondence between each page of the material and the handwritten memo will not be understood later.

  Therefore, in a conference room where a camera that captures participants' notes is installed, in response to the switching of the slide of the material displayed during the meeting, the captured image of the note at the start of the display of the slide and the display switching time A meeting material search support apparatus that extracts a difference from a captured image of a notebook, generates difference information including the difference area image data and a captured image at the time of display switching, and registers the slide ID and the difference information in association with each other. It has been proposed (see Patent Document 1). Thereby, it is possible to find out the conference material displayed when the memo is written in the note.

JP 2010-176603 A

  However, in the above prior art, a large-scale system including a computer device used for the progress of a conference, a display board for displaying conference materials, a camera for taking notes, and the like is necessary. Also, when searching for conference materials that were displayed when a memo was written, it is necessary to take a note with a camera and compare the captured image with the area image data included in the difference information. The process for identifying is complicated.

It is an object of the present invention to automatically store a captured image in association with output information output at the time of capturing .

The invention of claim 1 is an information processing apparatus for processing various types of information, and the information designated as an output target is being displayed on the display screen of the information processing apparatus, and the information processing apparatus When detecting the change to the specific state by the detection means for detecting whether or not the state has changed to the specific state, the position of the position different from the display screen is predetermined at the detection timing of the change. An acquisition unit that acquires both a subject image obtained by capturing an image of a certain subject by the imaging unit and the output information displayed on the display screen, and the subject image acquired by the acquisition unit and the output information are stored in association with each other. Storage control means for storing in the means .

According to the present invention, it is possible to automatically store a captured image in association with output information that was output at the time of imaging .

1 is an external perspective view of an information processing apparatus according to a first embodiment. It is a figure in the middle of closing a 1st cover (1st housing | casing) and a 2nd cover (2nd housing | casing). It is a block diagram which shows the functional structure of information processing apparatus. It is a flowchart which shows a 1st imaging | photography process. It is a figure for demonstrating the state with which the document image and the memo image were linked | related. It is a flowchart which shows a display process. (A) is an example of a memo image display screen. (B) is an example of a document image display screen. (A) is a figure which shows the state by which the information processing apparatus in 2nd Embodiment is accommodated in the main body cover. (B) is a side view showing a state in which the information processing apparatus is leaned against the main body cover. (C) is a perspective view showing a state in which the information processing apparatus is leaned against the main body cover. FIG. 11 is a diagram illustrating a state where a recording medium is placed in front of the information processing apparatus. It is a flowchart which shows a 2nd imaging | photography process. It is a flowchart which shows a 3rd imaging | photography process.

[First Embodiment]
First, a first embodiment of an information processing apparatus according to the present invention will be described with reference to FIGS. The present invention is not limited to the illustrated example.

FIG. 1 is an external perspective view of the information processing apparatus 1 according to the first embodiment. The information processing apparatus 1 is a tablet-type terminal device and includes a main body unit 10 and a binder 30.
The main body unit 10 is provided with a touch display unit 14 for screen display and touch input, an imaging unit 15 as an imaging unit, and the like.

  The binder 30 is configured by connecting a first cover 30a as a first housing and a second cover 30b as a second housing so that the hinge portion 30c can be opened and closed. The recording medium 40 is disposed on the first cover 30a, and the main body 10 is provided on the second cover 30b. The main body 10 and the recording medium 40 are detachably attached to the binder 30 with fasteners, hooks, and the like.

  In the present embodiment, the recording medium indicates a medium on which information to be photographed is recorded, and means a subject on which information is recorded. The recording medium also includes electronic paper that can display information electronically.

  The 1st cover 30a and the 2nd cover 30b are comprised by flat rectangular shape, and can be opened and closed so that a book and a notebook may be opened and closed by the hinge part 30c. FIG. 1 illustrates a state where the angle formed by the first cover 30a and the second cover 30b is 180 ° (open state). In this state, the user refers to the screen displayed on the touch display unit 14 and performs touch input, and enters characters, diagrams, and the like on the recording medium 40.

  The hinge portion 30c is provided with an opening / closing angle sensor 20 that detects an angle formed by the first cover 30a and the second cover 30b. The opening / closing angle sensor 20 includes a rotary encoder that detects rotation of the shaft of the hinge portion 30c, and a magnetic sensor that detects an opening / closing state using a hall element or the like.

  The recording medium 40 is a notebook or report sheet, a sheet on which a document is printed, a book, or the like used when a user takes notes. However, the recording medium 40 may be any medium on which information to be photographed is recorded, and the material is not limited to paper. The recording medium 40 may be a single sheet of paper or may be bound in a booklet.

  FIG. 2 is a view in the middle of closing the first cover 30 a and the second cover 30 b constituting the binder 30. The imaging unit 15 is provided at a position facing the first cover 30a of the main body unit 10, and an angle formed by the first cover 30a and the second cover 30b is a predetermined angle (for example, 70 °). In such a case, the recording medium 40 is imaged by the imaging unit 15. The predetermined angle is set in advance to an optimum angle for shooting, and shooting conditions such as focus and exposure are set in accordance with this angle. When the user closes the first cover 30a and the second cover 30b (when the angle formed by the first cover 30a and the second cover 30b is 0 °), the user passes a predetermined angle in the middle of closing, Shooting can be performed without the user being aware of it.

FIG. 3 is a block diagram illustrating a functional configuration of the information processing apparatus 1.
The information processing apparatus 1 includes a CPU (Central Processing Unit) 11 as a photographing control unit, a storage control unit, a detection unit, and a display control unit, a power supply unit 12, an operation unit 13, a touch display unit 14, and an imaging unit 15. A wireless communication unit 16, a RAM (Random Access Memory) 17, a ROM (Read Only Memory) 18, a storage unit 19, and an opening / closing angle sensor 20.

  The CPU 11 comprehensively controls the processing operation of each unit of the information processing apparatus 1 according to an operation signal input from the operation unit 13 or the touch display unit 14 or an instruction signal received by the wireless communication unit 16. Specifically, the CPU 11 reads out various processing programs stored in the ROM 18, develops them in the RAM 17, and performs various processes in cooperation with the programs.

The power supply unit 12 supplies power to each unit constituting the information processing apparatus 1.
The operation unit 13 includes operation keys such as a power key for turning on / off the power, and outputs an operation signal corresponding to each operation key to the CPU 11.

  The touch display unit 14 is configured by laminating a touch panel 14b on a display panel 14a serving as a display unit. The display panel 14a is composed of an LCD (Liquid Crystal Display) or the like, and performs screen display according to a display control signal from the CPU 11. The display panel 14a displays a document image as the first image. The touch panel 14b detects the position of a touch operation with a user's finger or the like, and outputs an operation signal corresponding to the position to the CPU 11. The touch panel 14b is an operation unit that receives an operation instruction from the user.

The imaging unit 15 is a digital camera that includes an optical lens, an imaging element, various sensors, and the like, and shoots a subject. The imaging unit 15 captures the recording medium 40 and generates a memo image as a second image.
The wireless communication unit 16 is a wireless communication module that connects to the Internet via a wireless LAN (Local Area Network) router and performs data communication.

The RAM 17 is a volatile memory. The RAM 17 temporarily stores various programs executed by the CPU 11 and data related to these various programs.
The ROM 18 is a read-only semiconductor memory that stores various programs and various data. The ROM 18 stores a photographing processing program 181 and a display processing program 182.

  The storage unit 19 is configured by an HDD (Hard Disk Drive), a nonvolatile memory, or the like, and stores information in a readable and writable manner. Specifically, the storage unit 19 stores electronic data of materials. The electronic data of the document may be data such as PDF (Registered Trademark; Portable Document Format) other than those created by presentation software that displays characters and charts in a slide format, and the data format is not limited.

  The opening / closing angle sensor 20 detects an angle formed by the first cover 30a and the second cover 30b, and outputs the detected angle to the CPU 11.

Based on the angle output from the opening / closing angle sensor 20, the CPU 11 detects that the first cover 30a and the second cover 30b are closed to a predetermined state.
The CPU 11 causes the imaging unit 15 to perform shooting at the timing when it is detected that the first cover 30a and the second cover 30b are closed to a predetermined state. Image data of a photographed image (memo image) is generated in a data format such as JPEG or TIFF.
The CPU 11 displays the memo image generated by the imaging unit 15 and the material displayed on the display panel 14a when it is detected that the first cover 30a and the second cover 30b are closed to a predetermined state. The images are associated with each other and stored in the storage unit 19. Image data obtained by screen-capturing the material image displayed on the display panel 14a is generated in a data format such as JPEG, GIF, or PNG.

The CPU 11 displays the display panel 14a from the storage unit 19 when an operation instruction for switching the display is input from the touch panel 14b in a state where the document image stored in the storage unit 19 is displayed on the display panel 14a. The memo image associated with the document image being read is read, and the read memo image is displayed on the display panel 14a.
When the operation instruction for switching the display is input from the touch panel 14b in a state where the memo image stored in the storage unit 19 is displayed on the display panel 14a, the CPU 11 displays the display panel 14a from the storage unit 19. The material image associated with the memo image being read is read, and the read material image is displayed on the display panel 14a.

Next, the operation in the information processing apparatus 1 will be described.
FIG. 4 is a flowchart illustrating a first photographing process executed in the information processing apparatus 1. This processing is realized by software processing in cooperation with the CPU 11 and the photographing processing program 181 stored in the ROM 18.

  First, the CPU 11 determines whether the opening / closing angle is equal to or smaller than a predetermined angle (for example, 70 °) based on the angle formed by the first cover 30a and the second cover 30b output by the opening / closing angle sensor 20. Is determined (step S1). That is, the CPU 11 determines whether or not the first cover 30a and the second cover 30b constituting the binder 30 are closed to a predetermined state.

  When the opening / closing angle is equal to or smaller than the predetermined angle (step S1; YES), that is, when the first cover 30a and the second cover 30b are closed to a predetermined state, the CPU 11 moves the imaging unit 15 to the predetermined position. Control is performed to capture the handwritten memo recorded in the recording medium 40, and the memo image generated by the imaging unit 15 is acquired (step S2). For example, the CPU 11 extracts a region of the recording medium 40 from the captured image output by the imaging unit 15 and performs a process of correcting the trapezoidal distortion on the extracted region. The CPU 11 stores the image data of the memo image in the RAM 17.

  Next, the CPU 11 captures the material image displayed on the display panel 14a of the touch display unit 14, and stores the material image (screen shot) (step S3). The CPU 11 stores the image data of the material image in the RAM 17.

  Next, the CPU 11 stores the memo image acquired in step S2 and the document image saved in step S3 in association with each other in the storage unit 19 (step S4).

  Next, the CPU 11 determines whether or not there is an instruction to end the shooting mode by the user's operation from the touch panel 14b (step S5).

  If the opening / closing angle is larger than the predetermined angle in step S1 (step S1; NO), or if there is no instruction to end the shooting mode in step S5 (step S5; NO), the process returns to step S1. The process is repeated.

  In step S5, when there is an instruction to end the shooting mode (step S5; YES), the first shooting process is ended.

  By the first photographing process, as shown in FIG. 5, the document image and the memo image are associated with each other. The document image and the memo image are not only associated one-to-one, but a plurality of memo images may be associated with one document image, or a plurality of document images are associated with one memo image. May be.

  In step S3, when the document image displayed on the display panel 14a is the same as the previously captured document image, the process of step S3 may be omitted.

  Next, display processing executed in the information processing apparatus 1 will be described with reference to FIG. This processing is realized by software processing in cooperation with the CPU 11 and the display processing program 182 stored in the ROM 18.

  First, the CPU 11 determines whether or not a memo image to be displayed is selected from the memo images stored in the storage unit 19 by a user operation from the touch panel 14b (step S11). When the memo image to be displayed is selected (step S11; YES), the CPU 11 reads the selected memo image from the storage unit 19 and displays the read memo image on the display panel 14a (step S12). ).

  Next, the CPU 11 determines whether or not a document image associated with the memo image displayed on the display panel 14a is stored in the storage unit 19 (step S13). When the document image associated with the currently displayed memo image is stored in the storage unit 19 (step S13; YES), the CPU 11 is associated with the currently displayed memo image by the user's operation from the touch panel 14b. It is determined whether or not the display of the document image is instructed (step S14). That is, the CPU 11 determines whether or not an operation instruction for switching the display is input.

  FIG. 7A shows an example of the memo image display screen 141 displayed on the display panel 14a. A memo image is displayed on the memo image display screen 141. When there is a document image associated with the memo image, a switching button B1 is displayed on the memo image display screen 141. The user can instruct the display of the document image associated with the currently displayed memo image by touching the switching button B1.

  In step S14, when the display of the document image associated with the currently displayed memo image is instructed (step S14; YES), the CPU 11 stores the document image associated with the currently displayed memo image from the storage unit 19. The read material image is read and displayed on the display panel 14a (step S15).

  Next, the CPU 11 determines whether or not a memo image associated with the document image displayed on the display panel 14a is stored in the storage unit 19 (step S16). When the memo image associated with the document image being displayed is stored in the storage unit 19 (step S16; YES), the CPU 11 is associated with the document image being displayed by the user's operation from the touch panel 14b. It is determined whether display of a memo image has been instructed (step S17). That is, the CPU 11 determines whether or not an operation instruction for switching the display is input.

  FIG. 7B shows an example of the document image display screen 142 displayed on the display panel 14a. A document image is displayed on the document image display screen 142. When there is a memo image associated with this document image, a switch button B2 is displayed on the document image display screen 142. The user can instruct the display of the memo image associated with the currently displayed document image by touching the switching button B2.

  When the display of the memo image associated with the currently displayed document image is instructed in step S17 (step S17; YES), the CPU 11 stores the memo image associated with the currently displayed document image from the storage unit 19. The read memo image is displayed on the display panel 14a (step S18), and the process proceeds to step S13.

  In step S11, when the memo image to be displayed is not selected (step S11; NO), the CPU 11 selects the display object among the document images stored in the storage unit 19 by the user's operation from the touch panel 14b. It is determined whether or not a material image has been selected (step S19). When the document image to be displayed is selected (step S19; YES), the CPU 11 reads the selected document image from the storage unit 19 and displays the read document image on the display panel 14a (step S20). ), The process proceeds to step S16.

  If no document image to be displayed is selected in step S19 (step S19; NO), the process returns to step S11.

  If the document image associated with the currently displayed memo image is not stored in the storage unit 19 in step S13 (step S13; NO), the document image associated with the currently displayed memo image is displayed in step S14. When not instructed (step S14; NO), when the memo image associated with the displayed document image is not stored in the storage unit 19 at step S16 (step S16; NO), or displayed at step S17. When the display of the memo image associated with the document image is not instructed (step S17; NO), the display process ends.

  As described above, according to the information processing apparatus 1 of the first embodiment, the imaging unit is detected at the timing when it is detected that the first cover 30a and the second cover 30b are closed to a predetermined state. 15 is photographed and displayed on the display panel 14a when it is detected that the memo image generated by the imaging unit 15 and the first cover 30a and the second cover 30b are closed to a predetermined state. Since the stored document image is associated and stored in the storage unit 19, there is a correspondence between the content (memo image) recorded on the recording medium 40 and the content (document image) displayed on the display panel 14a. It can be easily referenced.

  In addition, since shooting is performed at the timing when it is detected that the first cover 30a and the second cover 30b are closed to a predetermined state, it is possible to easily perform shooting.

  Further, in a state where the document image is displayed on the display panel 14a, it is possible to easily switch to the display of the memo image associated with the document image. Further, in a state where a memo image is displayed on the display panel 14a, it is possible to easily switch to display of a document image associated with the memo image.

  Note that a handwritten memo is photographed or scanned in advance to generate electronic data of the memo image, and the document (recording medium 40) is photographed in a state where the memo image is displayed on the display panel 14a. The image (second image) and the memo image (first image) displayed on the display panel 14a at the time of shooting may be stored in association with each other.

  Moreover, the detection method of the opening / closing angle of the 1st cover 30a and the 2nd cover 30b is not limited to said method. For example, an index is attached to a predetermined position of the first cover 30a, and the first cover 30a and the second cover 30b are determined based on the position of the index in the image captured by the imaging unit 15. It is good also as detecting that it was closed to this state. In addition, an acceleration sensor may be provided in the second cover 30b, and the direction of gravity may be detected by the acceleration sensor to detect the opening / closing angles of the first cover 30a and the second cover 30b.

[Second Embodiment]
Next, a second embodiment to which the present invention is applied will be described.
FIG. 8A is a diagram illustrating a state in which the information processing apparatus 1A according to the second embodiment is housed in the main body cover 50. FIGS. 8B and 8C illustrate the information processing apparatus 1A. FIG. 6 is a view showing a state where the main body cover 50 is leaned against the main body cover 50. In the information processing apparatus 1A, the same components as those of the information processing apparatus 1 shown in the first embodiment are denoted by the same reference numerals and description thereof is omitted.

  FIG. 9 is a diagram illustrating a state where the information processing apparatus 1A is leaned against the main body cover 50 and the recording medium 40 is placed in front of the information processing apparatus 1A. In the second embodiment, the recording medium 40 is configured as a booklet composed of a plurality of pages, and the recording medium 40 is photographed by the imaging unit 15 at the timing of turning the pages of the recording medium 40. When the recording medium 40 is placed so that the lower end of the information processing apparatus 1A is aligned with the upper end of the recording medium 40, the angle of view and the focus of the imaging unit 15 are set in advance so as to match the recording medium 40.

  The information processing apparatus 1A is different from the information processing apparatus 1 shown in the first embodiment in that the opening / closing angle sensor 20 and the binder 30 are not necessary. Since it is the same, FIG. 3 is used and description is abbreviate | omitted. Hereinafter, a configuration and processing characteristic of the second embodiment will be described.

The CPU 11 captures moving image data continuously captured by the imaging unit 15 and analyzes the moving image data to detect a page turning operation of the recording medium 40 being captured. While the image data output from the imaging unit 15 is changing, it is estimated that each page of the recording medium 40, the user's finger, and the like are moving. The CPU 11 continues to capture the moving image data captured by the imaging unit 15 and determines that the page turning operation has been performed when the image is still.
The CPU 11 causes the imaging unit 15 to perform shooting at the timing when the page turning operation of the recording medium 40 is detected.
The CPU 11 stores the memo image generated by the imaging unit 15 and the document image displayed on the display panel 14 a when the page turning operation of the recording medium 40 is detected in the storage unit 19 in association with each other.

Next, the operation in the information processing apparatus 1A will be described.
FIG. 10 is a flowchart showing the second photographing process executed in the information processing apparatus 1A. This processing is realized by software processing in cooperation with the CPU 11 and the photographing processing program 181 stored in the ROM 18. This process assumes a case where a user shoots handwritten memos composed of a plurality of pages recorded on the recording medium 40 while displaying the materials on the display panel 14a and confirming them after the end of the meeting.

  First, when an instruction to start the shooting mode is given by the user's operation from the touch panel 14b, the CPU 11 controls the imaging unit 15 to cause the handwritten memo recorded in the recording medium 40 to be shot. A memo image (still image) generated by the imaging unit 15 is acquired (step S21). The CPU 11 stores the image data of the memo image in the RAM 17.

  Next, the CPU 11 captures the material image displayed on the display panel 14a and stores the material image (step S22). The CPU 11 stores the image data of the material image in the RAM 17.

  Next, the CPU 11 stores the memo image acquired in step S21 and the document image stored in step S22 in association with each other in the storage unit 19 (step S23).

  Next, the CPU 11 determines whether or not an instruction to switch the document image displayed on the display panel 14a has been performed by the user's operation from the touch panel 14b (step S24). When the instruction to switch the document image is detected (step S24; YES), the CPU 11 switches the document image to the next page on the display panel 14a based on the electronic data of the document stored in the storage unit 19. It is displayed (step S25).

  In step S24, when the instruction to switch the material image is not detected (step S24; NO), the CPU 11 reads the handwritten memo (recording medium 40) based on the moving image data continuously captured from the imaging unit 15. It is determined whether or not a page turning operation has been detected (step S26).

  When the page turning operation of the handwritten memo is detected (step S <b> 26; YES), the CPU 11 controls the image capturing unit 15 to photograph the handwritten memo recorded in the recording medium 40, and is generated by the image capturing unit 15. The obtained memo image (still image) is acquired (step S27). The CPU 11 stores the image data of the memo image in the RAM 17.

  Next, the CPU 11 captures the material image displayed on the display panel 14a and stores the material image (step S28). The CPU 11 stores the image data of the material image in the RAM 17.

  Next, the CPU 11 stores the memo image acquired in step S27 and the document image stored in step S28 in association with each other in the storage unit 19 (step S29).

  Next, the CPU 11 determines whether or not there is an instruction to end the shooting mode by an operation from the touch panel 14b of the user (step S30).

  In step S26, when the page turning operation of the handwritten memo is not detected (step S26; NO), in the case where there is no instruction to end the shooting mode in step S30 (step S30; NO), or after step S25, Returning to step S24, the process is repeated.

  If there is an instruction to end the shooting mode in step S30 (step S30; YES), the second shooting process ends.

  In step S28, when the document image displayed on the display panel 14a is the same as the previously captured document image, the process of step S28 may be omitted.

  The display process when displaying the document image and the memo image associated with each other by the second photographing process is the same as the display process described in the first embodiment.

  As described above, according to the information processing apparatus 1 </ b> A of the second embodiment, the imaging unit 15 performs imaging at the timing when the page turning operation of the recording medium 40 is detected, and is generated by the imaging unit 15. Since the memo image and the document image displayed on the display panel 14a when the page turning operation of the recording medium 40 is detected are associated and stored in the storage unit 19, the contents recorded on the recording medium 40 (memo The correspondence between the image) and the content (document image) displayed on the display panel 14a can be easily referred to.

  Further, since shooting is performed at the timing when the page turning operation of the recording medium 40 is detected, shooting can be performed easily.

  In addition, the electronic data of the memo image is generated by photographing or scanning the handwritten memo in advance, and the booklet-shaped material (recording medium 40) including a plurality of pages is displayed in a state where the memo image is displayed on the display panel 14a. Images may be taken according to the page turning operation of the material, and the material image obtained by the photographing may be stored in association with the memo image displayed on the display panel 14a at the time of photographing.

  Further, the method of detecting the page turning operation is not limited to the above method. For example, when a mark for identifying a page is attached to a predetermined position of each page of the recording medium 40 in advance and the mark is detected from a predetermined position in the captured image, the page turning operation is performed. It may be judged that it has been broken.

[Third Embodiment]
Next, a third embodiment to which the present invention is applied will be described.
The information processing apparatus 1B according to the third embodiment is different from the information processing apparatus 1 according to the first embodiment in that the opening / closing angle sensor 20 and the binder 30 are unnecessary, but other configurations Since it is the same as that of the information processing apparatus 1, FIG. 3 is used and description is abbreviate | omitted. Hereinafter, a configuration and processing characteristic of the third embodiment will be described.

The CPU 11 detects an instruction to switch a document image displayed on the display panel 14a (an instruction to switch a document image consisting of a plurality of pages to the next page) based on a user operation from the touch panel 14b.
The CPU 11 causes the imaging unit 15 to perform shooting at the timing when the instruction to switch the document image displayed on the display panel 14a is detected.
The CPU 11 stores the memo image generated by the imaging unit 15 and the document image displayed on the display panel 14a when the document image switching instruction is detected in the storage unit 19 in association with each other.

Next, the operation in the information processing apparatus 1B will be described.
FIG. 11 is a flowchart showing a third photographing process executed in the information processing apparatus 1B. This processing is realized by software processing in cooperation with the CPU 11 and the photographing processing program 181 stored in the ROM 18.

  First, the CPU 11 determines whether or not an instruction to switch the document image displayed on the display panel 14a has been performed by a user operation from the touch panel 14b (step S31). When a document image switching instruction is detected (step S31; YES), the CPU 11 controls the imaging unit 15 to capture a handwritten memo recorded in the recording medium 40 and is generated by the imaging unit 15. A memo image is acquired (step S32). The CPU 11 stores the image data of the memo image in the RAM 17.

  Next, the CPU 11 captures the material image displayed on the display panel 14a and stores the material image (step S33). The CPU 11 stores the image data of the material image in the RAM 17.

  Next, the CPU 11 stores the memo image acquired in step S32 and the document image stored in step S33 in association with each other in the storage unit 19 (step S34).

  Next, based on the electronic data of the material stored in the storage unit 19, the CPU 11 switches the material image to the next page and displays it on the display panel 14a (step S35). Since the document image to be associated with the memo image is acquired in step S33 before the document image is switched in step S35, the memo image and the document image displayed when the memo is entered can be associated.

  Next, the CPU 11 determines whether or not there is an instruction to end the shooting mode by the user's operation from the touch panel 14b (step S36).

  If no instruction to switch the document image is detected in step S31 (step S31; NO), or if there is no instruction to end the shooting mode in step S36 (step S36; NO), the process returns to step S31. The process is repeated.

  In step S36, when there is an instruction to end the shooting mode (step S36; YES), the third shooting process ends.

  The display process when displaying the document image and the memo image associated with each other by the third imaging process is the same as the display process described in the first embodiment.

  As described above, according to the information processing apparatus 1B of the third embodiment, the imaging unit 15 performs imaging at the timing when the instruction to switch the document image displayed on the display panel 14a is detected, and the imaging unit 15 is associated with the document image displayed on the display panel 14a when the instruction to switch the document image is detected, and is stored in the storage unit 19, so that it is recorded in the recording medium 40. The correspondence between the contents (memo image) and the contents (document image) displayed on the display panel 14a can be easily referred to.

  In addition, since the photographing is performed at the timing when the instruction to switch the document image displayed on the display panel 14a is detected, the photographing can be easily performed.

  In the third embodiment, the case where the document image displayed on the display panel 14a is switched based on the user's operation from the touch panel 14b has been described. However, the third embodiment is acquired from the external device via the wireless communication unit 16. When the document image is passively displayed on the display panel 14a, it may be determined that the document image has been switched when the content displayed on the display panel 14a changes. In this case, since the display content of the display panel 14a is switched regardless of the user's will, a memo image taken at the timing when the document image is switched, a document image obtained by capturing the display panel 14a, Deviation occurs between the two. Therefore, the first document image may be acquired in advance, and the first document image may be associated with the memo image taken at the timing of switching to the second document image. Then, the second material image obtained by capturing the display panel 14a at the timing of switching to the second material image and the memo image taken at the timing of switching to the third material image are sequentially provided. You just need to associate them.

  The description in each of the above embodiments is an example of the information processing apparatus according to the present invention, and the present invention is not limited to this. The detailed configuration and detailed operation of each part constituting the apparatus can be changed as appropriate without departing from the spirit of the present invention.

  For example, in each of the above embodiments, the document image and the memo image associated with each other are switched and displayed on the display panel 14a. However, the document image and the memo image associated with each other are displayed on the display panel 14a. It may be displayed side by side on the same screen. Alternatively, print data may be output to the printer via the wireless communication unit 16 so that the document image and the memo image associated with each other are printed on the same page.

  Further, the material image displayed on the display panel 14 a is not limited to one stored in the storage unit 19 in advance, and is acquired from an external device such as a Web display screen acquired from the Web server via the wireless communication unit 16. It may be.

  In the above description, an example in which a ROM is used as a computer-readable medium storing a program for executing each process is disclosed, but the present invention is not limited to this example. As other computer-readable media, a non-volatile memory such as a flash memory and a portable recording medium such as a CD-ROM can be applied. A carrier wave may be applied as a medium for providing program data via a communication line.

Although the embodiments of the present invention have been described, the scope of the present invention is not limited to the above-described embodiments, but includes the scope of the invention described in the claims and the equivalents thereof.
The invention described in the scope of claims attached to the application of this application will be added below. The item numbers of the claims described in the appendix are as set forth in the claims attached to the application of this application.
[Appendix]
<Claim 1>
Display means for displaying the first image;
Imaging means for capturing a recording medium and generating a second image;
Photographing control means for causing the imaging means to perform photographing at a predetermined timing;
Storage control means for associating and storing the second image generated by the imaging means and the first image displayed on the display means at the predetermined timing in the storage means;
An information processing apparatus comprising:
<Claim 2>
A first housing in which the recording medium is disposed;
A second housing which is connected to the first housing by a hinge portion so as to be openable and closable and provided with a main body portion;
Detecting means for detecting that the first casing and the second casing are closed to a predetermined state;
With
The imaging means is provided at a position facing the first housing of the main body,
The predetermined timing is a timing at which the detection unit detects that the first casing and the second casing are closed to the predetermined state.
The information processing apparatus according to claim 1.
<Claim 3>
The recording medium is configured in a booklet shape including a plurality of pages,
Detecting means for detecting a page turning operation of the recording medium;
The predetermined timing is a timing at which a page turning operation of the recording medium is detected by the detection unit.
The information processing apparatus according to claim 1.
<Claim 4>
Detecting means for detecting an instruction to switch the first image displayed on the display means;
The predetermined timing is a timing at which an instruction to switch the first image displayed on the display unit is detected by the detection unit.
The information processing apparatus according to claim 1.
<Claim 5>
An operation means for receiving an operation instruction by a user;
When an operation instruction for switching the display is input from the operation unit in a state where the first image stored in the storage unit is displayed on the display unit, the display unit displays the display unit on the display unit. The second image associated with the first image being read is read out, the read second image is displayed on the display means, and the second image stored in the storage means is displayed on the display means. When an operation instruction for switching the display is input from the operation unit in the displayed state, the first image associated with the second image displayed on the display unit is read from the storage unit Display control means for displaying the read first image on the display means;
Comprising
The information processing apparatus according to any one of claims 1 to 4.
<Claim 6>
Computer
Display control means for displaying the first image on the display means;
Photographing control means for causing an imaging means for photographing a recording medium to generate a second image to perform photographing at a predetermined timing;
Storage control means for associating and storing the second image generated by the imaging means and the first image displayed on the display means at the predetermined timing in the storage means;
Program to function as.

1 Information Processing Device 1A Information Processing Device 1B Information Processing Device 10 Main Body 11 CPU
12 power supply unit 13 operation unit 14 touch display unit 14a display panel 14b touch panel 15 imaging unit 16 wireless communication unit 17 RAM
18 ROM
19 Storage unit 20 Opening / closing angle sensor 30 Binder 30a First cover 30b Second cover 30c Hinge unit 40 Recording medium 50 Main body cover 181 Shooting processing program 182 Display processing program

Claims (8)

An information processing apparatus that processes various types of information,
Detecting means for detecting whether the information designated as the output target is being displayed on the display screen of the information processing apparatus and the information processing apparatus has changed to a specific state;
When it is detected that the detection unit has changed to the specific state, a subject image obtained by imaging the subject at a predetermined position at a position different from the display screen at the change detection timing, and the display screen Acquisition means for acquiring together the output information displayed in
Storage control means for associating and storing the subject image acquired by the acquisition means and the output information in a storage means;
An information processing apparatus comprising: Determining means for determining whether or not there is output information associated with the subject image when the subject image is output;
When it is determined that there is output information related by the determination means, output control means for controlling to output the output content based on the output information;
The information processing apparatus according to claim 1 , further comprising: Determining means for determining whether or not there is output information associated with the subject image when the subject image is output;
When it is determined that there is output information related by the determination unit, an identification unit that outputs output identification information indicating that the output information exists;
The information processing apparatus according to claim 1 , further comprising: When the output identification information output by the identification means is selected, output control means for controlling to output the output content based on the output information;
The information processing apparatus according to claim 3 , further comprising: Second determination means for determining whether or not there is a subject image associated with the output information when outputting the output content based on the output information;
A second output control means for controlling to output a captured image when it is determined by the second determination means that there is a related subject image;
The information processing apparatus according to any one of claims 1-4, characterized by comprising a. Second determination means for determining whether or not there is a subject image associated with the output information when outputting the output content based on the output information;
A second identification unit that outputs second output identification information indicating that the associated subject image exists when the second determination unit determines that there is an associated subject image;
The information processing apparatus according to any one of claims 1-4, characterized by comprising a. Second output control means for controlling to output the associated subject image when the second output identification information output by the second identification means is selected;
The information processing apparatus according to claim 6 , further comprising: A program for controlling a computer of an information processing apparatus that processes various types of information,
The computer,
Detecting means for detecting whether the information designated as the output target is being displayed on the display screen of the information processing apparatus and the information processing apparatus has changed to a specific state;
When it is detected that the detection unit has changed to the specific state, a subject image obtained by imaging the subject at a predetermined position at a position different from the display screen at the change detection timing, and the display screen Acquisition means for acquiring together with the output information displayed in
Storage control means for associating and storing the subject image acquired by the acquisition means and the output information in a storage means;
A program designed to function as

Images