JP6813079B2 - Information processing equipment, recording systems, and programs - Google Patents

Description

The present invention relates to a program capable of causing an image recording device to execute a recording process for content data acquired from a service providing device.

Conventionally, there has been known a program capable of recording an image represented by data acquired from a service providing device on a sheet in an image recording device. For example, the program disclosed in Patent Document 1 includes a recording process for recording a character string indicating a URL on a sheet in response to a page obtained from a WEB server containing a character string indicating the URL. The image recording device can be made to execute the recording process of recording the page indicated by the URL on the sheet.

Japanese Unexamined Patent Publication No. 2013-134618

However, the page may include not only character string data but also data in various data formats such as image data, document data, and spreadsheet data. In addition, a plurality of data having different data formats may be mixed on one page.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a program capable of executing recording processing with appropriate set values according to the data format of the content data included in the layout definition data. To provide.

(1) The program described in the present specification includes a display unit, a storage unit, an operation unit that accepts a user operation for touching the display screen of the display unit, and a communication unit, and records an image on a sheet. It can be executed by a computer that wirelessly communicates with an image recording device and a service providing device capable of executing the recording process to be performed through the communication unit. The storage unit stores format information indicating a data format of content data that can be acquired from the service providing device in association with a set value used for the recording process. Then, the program receives layout data acquisition processing for receiving layout definition data including layout information indicating the arrangement of content images indicating the content data from the service providing device through the communication unit, and the content image according to the layout information. The layout display process of displaying the layout image in which the above is arranged is displayed on the display unit in a state where the user operation for selecting the content image can be received by the operation unit, and the content data is received from the service providing device through the communication unit. In response to the reception of the content data acquisition process and the user operation of touching the position of the content image by the operation unit, the recording process for the content data indicated by the content image is converted into the data format of the content data. The operation is a preview display process for displaying a preview image representing the result when executed according to the corresponding set value on the display unit, and a user operation for instructing execution of the recording process for which the result is represented by the preview image. In response to the reception by the unit, the computer is made to execute an output process of transmitting the recording instruction information instructing the execution of the recording process to the image recording device through the communication unit.

According to the above configuration, the recording process for the content data is executed according to the set value corresponding to the data format of the selected content data. That is, the recording process is executed with an appropriate set value according to the data format. The setting value of the recording process is, for example, a value indicating the size and orientation of the sheet, the resolution, the magnification, the number of copies, and the like.

Further, according to the selection of the content data on the layout image, the result of the recording process executed for the content data under the corresponding execution conditions is preview-displayed. As a result, the user can easily select the content to be recorded from the layout images laid out for easy viewing, and after confirming the result of the recording process, input the recording instruction.

The "result of the recording process" in the present specification refers to an image when it is assumed that the recording process is executed without actually requiring the recording process to be executed. That is, the preview image is an image image on the assumption that the recording process has been executed, and indicates, for example, a state in which the image represented by the content data is represented on a sheet having a predetermined size.

(2) Preferably, in the layout data acquisition process, the program obtains identification information for identifying the content data and thumbnail image data corresponding to the content data in an image format from the service providing device through the communication unit. Further received, in the layout display process, the content data in a data format different from the image format includes the thumbnail image indicated by the thumbnail image data as the content image corresponding to the content data in the image format. As the content image corresponding to the above, the layout image includes a format image corresponding to the data format and the identification information for identifying the content data.

According to the above configuration, the user can confirm the content of the content data in the image format on the layout image. As a result, the user can quickly select the desired content data.

(3) Preferably, in the layout display process, the program uses the format image and the identification information as the content image corresponding to the content data in a data format different from the image format and in which the recording process can be executed. As the content image corresponding to the content data in a data format different from the image format and in which the recording process cannot be executed, the non-corresponding image indicating that the recording process cannot be executed and the identification information are included in the layout image. Include in layout image.

According to the above configuration, the user can be made to recognize at a glance whether or not the recording process can be executed, so that the user can select the desired content data more quickly.

(4) Preferably, the program causes the computer to further execute a thumbnail generation process of generating thumbnail image data corresponding to the content data acquired in the content data acquisition process and storing the data in the storage unit. In the layout display processing, the thumbnail image indicated by the thumbnail image data and the identification information are included in the layout image as the content image corresponding to the content data for which the thumbnail image data has already been generated.

According to the above configuration, the user can confirm the content of the content data on the layout image even for the content data that is not in the image format. As a result, the user can select the desired content data more quickly.

(5) For example, the layout definition data includes text data and layout information indicating the arrangement of text indicated by the text data. In the layout display process, the program arranges the text at a position on the layout image indicated by the layout information.

(6) Preferably, the storage unit further stores the text setting value, which is the setting value used for the recording process for the text. Then, the program performs the recording process for the extracted text in the preview display process executed in response to the user operation instructing the extraction of the text from the layout definition data to be received by the operation unit. The preview image showing the result when the text is executed according to the text setting value is displayed on the display unit.

According to the above configuration, the recording process can be executed on the text directly included in the layout definition data. The text included in the layout definition data refers to a character string reflected in the layout image, and does not include tags, metadata, or the like.

(7) Preferably, the program performs a plurality of the contents in the preview display process executed in response to the operation unit accepting the user operation for selecting the plurality of the contents images included in the layout image. The display unit displays the preview image showing the result when the recording process for each data is executed according to the set value corresponding to the data format of the content data.

According to the above configuration, since the recording process for a plurality of content data included in the layout image can be collectively executed, the recording process for a large number of content data can be realized with a small number of operations.

(8) Preferably, the storage unit further stores the layout image setting value, which is the setting value used for the recording process for the layout image. Then, the program executes the recording process for the layout image according to the layout image setting value in the preview display process executed in response to the user operation for selecting the layout image being received by the operation unit. The preview image showing the result in the case of the above is displayed on the display unit.

According to the above configuration, not only the content data alone but also the layout image in which the content image is laid out can be recorded on the sheet.

(9) Preferably, in the preview display process, the program selects only the content image corresponding to the content data in a data format in which the recording process can be executed, among the content images included in the layout image. , Included in the preview image.

According to the above configuration, the content image of the content data whose recording process cannot be executed is not reflected in the preview image and is not recorded in the sheet.

(10) For example, the program causes the computer to further execute an acquisition process for acquiring authentication information indicating that the service providing device has the acquisition authority for data stored in the service providing device, and further executes the layout data acquisition process and the content. In the data acquisition process, the layout definition data and the content data are received from the service providing device through the communication unit as a response of the authentication information transmitted to the service providing device through the communication unit.

(11) For example, the program causes the computer to further execute a service selection process for the user to select one of the plurality of service providing devices as the service providing device from which the layout definition data is acquired. The computer is made to execute the acquisition process in response to the reception of the user operation for selecting the specific service providing device from the plurality of the service providing devices by the operation unit.

(12) Preferably, the program receives the first thumbnail image data corresponding to the layout definition data stored in the service providing device from the service providing device through the communication unit, and the layout data list acquisition process. A plurality of the layout identification images are displayed on the display unit in a state in which the operation unit can accept a user operation for selecting one of the layout identification images corresponding to the layout definition data stored in the service providing device. In response to the operation unit accepting the layout data selection process and the user operation of touching the position of the layout identification image, the layout definition data corresponding to the layout identification image is transmitted to the service providing device through the communication unit. Using the layout data acquisition process received from the above and the layout definition data acquired in the layout data acquisition process, a second thumbnail image data corresponding to the layout identification image displayed in the layout display process is generated. The computer is made to execute the thumbnail generation process to be stored in the storage unit. Then, in the layout data selection process, the program uses the first thumbnail indicated by the first thumbnail image data as the layout identification image corresponding to the layout definition data for which the second thumbnail image data has not yet been generated. As the layout identification image corresponding to the layout definition data in which the image is displayed on the display unit and the second thumbnail image data has already been generated, the second thumbnail image indicated by the second thumbnail image data is displayed on the display unit. To display.

According to the above configuration, the user can confirm the contents of each layout definition data in the layout identification image displayed in the list. As a result, the user can quickly select the desired layout definition data.

(13) The information processing apparatus described in the present specification includes a display unit, a storage unit, an operation unit that receives a user operation for touching the display screen of the display unit, a communication unit, and a control unit. The storage unit stores format information indicating a data format of content data that can be acquired from the service providing device in association with a set value used for the recording process. The communication unit can wirelessly communicate with an image recording device and a service providing device capable of executing a recording process of recording an image on a sheet. Then, the control unit receives layout data acquisition processing for receiving layout definition data including layout information indicating the arrangement of content images indicating the content data from the service providing device through the communication unit, and the content according to the layout information. The layout display process of displaying the layout image on which the image is arranged is displayed on the display unit in a state where the user operation for selecting the content image can be accepted by the operation unit, and the content data is transmitted from the service providing device through the communication unit. In response to the reception of the content data acquisition process to be received and the user operation of touching the position of the content image by the operation unit, the recording process for the content data indicated by the content image is performed in the data format of the content data. The user operation for instructing the execution of the preview display process for displaying the preview image representing the result when the result is executed according to the set value corresponding to the above on the display unit and the recording process for which the result is represented by the preview image is described. In response to the reception by the operation unit, the output process of transmitting the recording instruction information instructing the execution of the recording process to the image recording device through the communication unit is executed.

(14) The recording system described in the present specification wirelessly communicates with a display unit, a storage unit, an operation unit that accepts a user operation for touching the display screen of the display unit, an image recording device, and a service providing device. An information processing device including a communication unit and a first control unit, a recording unit capable of executing a recording process for recording an image on a sheet, an image recording device including a second communication unit and a second control unit. To be equipped. The storage unit stores the format information indicating the data format of the content data that can be acquired from the service providing device and the condition information indicating the execution condition of the recording process in association with each other. Then, the first control unit receives layout data acquisition processing for receiving layout definition data including layout information indicating the arrangement of content images indicating the content data from the service providing device through the first communication unit, and the layout. The layout display process of displaying the layout image in which the content image is arranged according to the information on the display unit in a state where the user operation for selecting the content image can be accepted by the operation unit, and the service provision through the first communication unit. In response to the reception of the content data acquisition process for receiving the content data from the device and the user operation for touching the position of the content image by the operation unit, the recording process for the content data indicated by the content image is performed. A preview display process for displaying a preview image showing the result when executed according to the set value corresponding to the data format of the content data is displayed on the display unit, and an execution of the recording process for which the result is represented by the preview image. In response to the reception of the user operation to be instructed by the operation unit, the output process of transmitting the recording instruction information instructing the execution of the recording process to the image recording device through the first communication unit is executed. Further, the second control unit causes the recording unit to execute the recording process indicated by the recording instruction information in response to receiving the recording instruction information from the information processing device through the second communication unit.

According to the present invention, after the user is made to confirm the result of the recording process executed according to the appropriate setting according to the data format in the preview image, the recording process in which the result is represented by the preview image is executed.

FIG. 1 is a block diagram of the recording system 100 according to the embodiment. FIG. 2 is an example of the format information and the set value stored in the data storage area 62B. FIG. 3A is an example of layout definition data, and FIG. 3B is an example of subject information. FIG. 4 is a flowchart of the notebook printing process according to the first embodiment. FIG. 5 is a flowchart of the note list display process. FIG. 6 is a flowchart of the note display process. 7A and 7B are display examples of the display unit 53, in which FIG. 7A shows a service selection screen and FIG. 7B shows an account information input screen. 8A and 8B are display examples of the note selection screen on the display unit 53, in which FIG. 8A includes a first thumbnail image 111 and FIG. 8B includes a second thumbnail image 113. 9A and 9B are display examples of a notebook display screen on the display unit 53, in which FIG. 9A includes a format image 117 and FIG. 9B includes a thumbnail image 118. 10A and 10B are display examples of a preview screen on the display unit 53, in which FIG. 10A includes a preview image 121 of content data in an image format, and FIG. 10B includes a preview image 122 of content data different from the image format. 11A and 11B are display examples of the display unit 53, in which FIG. 11A shows a note preview screen and FIG. 11B shows a text preview screen. FIG. 12 is a diagram illustrating a sharing function realized by the shared API. FIG. 13 is a flowchart of the notebook printing process according to the second embodiment. FIG. 14 is a flowchart of the token acquisition process. FIG. 15 is another example of layout definition data. 16A and 16B are display examples of the display unit 53, in which FIG. 16A shows a preview screen including the layout image 125, and FIG. 16B shows a preview screen including the layout image 126. FIG. 17 is a flowchart of the note printing process according to the modified example of the second embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings as appropriate. It goes without saying that the embodiments described below are merely examples of the present invention, and the embodiments of the present invention can be appropriately changed without changing the gist of the present invention.

FIG. 1 is a schematic view of the recording system 100 according to the present embodiment. The recording system 100 shown in FIG. 1 is composed of a multifunction device 10, a mobile terminal 50, and service providing devices 70 and 80. On the other hand, the service providing device 90 will be described later in the second embodiment. The multifunction device 10, the mobile terminal 50, and the service providing devices 70 and 80 can communicate with each other via the communication network 102. Specific examples of the communication network 102 are not particularly limited, but may be, for example, the Internet, a mobile communication network, a wired LAN (abbreviation of Local Area Network), a wireless LAN, or a combination thereof.

[Multifunction device 10]
As shown in FIG. 1, the multifunction device 10 includes a printer unit 11, a scanner unit 12, a display unit 23, an operation unit 24, a communication unit 25, a CPU 31, a storage unit 32, and a communication bus 33. Mainly prepared. The components constituting the multifunction device 10 are connected to each other via the communication bus 33. The multifunction device 10 is an example of an image recording device.

[Printer unit 11, scanner unit 12]
The printer unit 11 executes a recording process of recording an image represented by image data on a recording sheet which is an example of a sheet. The recording method of the printer unit 11 is not particularly limited, but for example, a known method such as an inkjet method or an electrophotographic method can be adopted. The scanner unit 12 executes a scanning process of reading the image recorded on the recording paper and generating image data. The multifunction device 10 may further have a fax function for transmitting and receiving faxes, a copy function for reading an image recorded on a recording paper and recording it on another recording paper, and the like.

[Display unit 23]
The display unit 23 includes a display screen for displaying various information. The specific configuration of the display unit 23 is not particularly limited, but for example, a liquid crystal display (abbreviation of Liquid Crystal Display), an organic EL display (abbreviation of Organic Electro-Luminence Display), and the like can be adopted.

[Operation unit 24]
The operation unit 24 receives an operation of a user who selects an object displayed on the display screen of the display unit 23. Specifically, the operation unit 24 has, for example, a push button, and outputs various operation signals associated with the pressed push button to the CPU 31. Further, the operation unit 24 may have a film-like touch sensor superimposed on the display screen of the display unit 23. That is, the display unit 23 may be configured as a touch panel display. As the touch sensor, a well-known method such as a capacitance method or a resistance film method can be adopted.

The "object" refers to an image that can be selected by the user by operating the operation unit 24. As an example, the object is a character string displayed on the display unit 23, and one of the objects is highlighted by pressing the direction key of the operation unit 24, and high by pressing the enter button of the operation unit 24. Lighted objects may be selected. As another example, when the operation unit 24 is a touch panel, the object may be an icon, a button, a link, or the like displayed on the display unit 23, and the object displayed at the touch position may be selected.

[Communication unit 25]
The communication unit 25 is an interface for communicating with an external device through the communication network 102. That is, the multifunction device 10 outputs various information to the mobile terminal 50 or the service providing devices 70 and 80 through the communication unit 25, and receives various data or various information from the mobile terminal 50 or the service providing devices 70 and 80 through the communication unit 25. To do. The specific communication procedure of the communication unit 25 is not particularly limited, but for example, Wi-Fi (registered trademark of Wi-Fi Alliance) can be adopted.

[CPU31]
The CPU (abbreviation of Central Processing Unit) 31 controls the overall operation of the multifunction device 10. The CPU 31 acquires and executes various programs to be described later from the storage unit 32 based on various information output from the operation unit 24 and various information acquired from the external device through the communication unit 25. That is, the CPU 31 and the storage unit 32 form an example of the control unit.

[Storage 32]
The storage unit 32 has a program storage area 32A and a data storage area 32B. The OS (abbreviation of Operating System) 34 and the control program 35 are stored in the program storage area 32A. The control program 35 may be a single program or an aggregate of a plurality of programs. Data or information necessary for executing the control program 35 is stored in the data storage area 32B. As shown in FIG. 2, the data storage area 32B stores the format information and the set value in association with each other.

The format information is information indicating the data format of the content data, the text data, and the layout definition data described later. The format information in the example of FIG. 2 is "jpg" which is an example of the data format of the image data, "pdf" which is an example of the data format different from the image format, "txt" which is an example of the data format of the text data, and the layout. "Xml" which is an example of the data format of the definition data is shown. However, specific examples of formal information are not limited to these. Hereinafter, the data format of the image data will also be described as an image format.

The set value is a value indicating an execution condition when the multifunction device 10 executes the recording process for the data of the corresponding data format. The set values in the example of FIG. 2 are a value indicating "paper size" (for example, L plate, A4, etc.), a value indicating "paper type" (for example, glossy paper, plain paper, etc.), and "number of colors". It is a value to be shown (for example, full color, 256 colors, monochrome, etc.). However, specific examples of the set values are not limited to these, and may include values indicating the orientation, resolution, magnification, number of copies, and the like of the paper.

It should be noted that "data" and "information" in the present specification are common in that they are bits or bit strings that can be handled by a computer. "Data" refers to data that can be handled without the computer considering the meaning and content of each bit. On the other hand, "information" refers to information that branches the operation of a computer depending on the meaning of each bit. Further, the "instruction" is a control signal for prompting the destination device to perform the next operation, may include information, and has the property of information itself. There is also.

In addition, even if the format (for example, text format, binary format, flag format, etc.) is changed for each computer, the "data" and "information" are the same data and information as long as they are recognized as having the same meaning. Treated as. For example, information indicating that there are "two" is held as text format information "0x32" in ASCII code on one computer, and as binary format information "10" in binary notation on another computer. May be done.

However, the above distinction between "data" and "information" is not strict, and exceptional handling is permitted. For example, the data may be temporarily treated as information, or the information may be temporarily treated as data. Further, what is treated as data in one device may be treated as information in another device. Furthermore, information may be extracted from the data, or data may be extracted from the information.

The storage unit 32 includes, for example, a RAM (abbreviation of Random Access Memory), a ROM (abbreviation of Read Only Memory), an EEPROM (abbreviation of Electrically Erasable Program Read-Only Memory), a CPU (abbreviation of Memory), and an HDD (abbreviation of HDD). It is composed of a buffer or the like or a combination thereof.

The storage unit 32 may be a storage medium that can be read by a computer. A computer-readable storage medium is a non-transitory medium. In addition to the above examples, the non-transitory medium also includes recording media such as a CD-ROM and a DVD-ROM. The non-transitory medium is also a tangible medium. On the other hand, an electric signal that carries a program downloaded from a server on the Internet is a computer-readable signal medium, which is a kind of computer-readable medium, but is a storage that can be read by a non-transitory computer. Not included in the medium.

The program stored in the program storage area 32A is executed by the CPU 31. However, in the present specification, the operation of each program may be described by omitting the CPU 31. That is, in the following description, the description to the effect that "the program A executes the process A" may refer to "the CPU 31 executes the process A described in the program A". The same applies to the mobile terminal 50 described later.

The OS 34 provides an API (abbreviation for Application Programming Interface) for controlling the printer unit 11, the scanner unit 12, the display unit 23, the operation unit 24, the communication unit 25, and the like, which are the hardware constituting the multifunction device 10. It is a basic program. That is, each of the above programs controls each hardware by calling the API provided by the OS 34. However, in this specification, the operation of each program will be described by omitting OS34. That is, in the following description, the description to the effect that "program B controls hardware C" may refer to "program B controls hardware C through the API of OS34". The same applies to the mobile terminal 50 described later.

[Mobile terminal 50]
As shown in FIG. 1, the mobile terminal 50 mainly includes a display unit 53, an operation unit 54, a communication unit 55, a CPU 61, a storage unit 62, and a communication bus 63. The display unit 53, the operation unit 54, the communication unit 55, the CPU 61, the storage unit 62, and the communication bus 63 included in the mobile terminal 50 are the display unit 23, the operation unit 24, the communication unit 25, and the CPU 31 included in the multifunction device 10. Since it is common to the storage unit 32 and the communication bus 33, the description thereof will be omitted again. The CPU 61 and the storage unit 62 are examples of control units.

The operation unit 54 of the mobile terminal 50 includes a film-like touch sensor superimposed on the display screen of the display unit 53. Then, the operation unit 54 accepts a user operation of touching the display screen of the display unit 53. Further, the operation unit 54 outputs position information indicating a position on the display screen touched by the user. The position information can be expressed, for example, as coordinates (x, y) on the xy plane with the upper left edge of the display screen as the origin, the right direction as the positive direction of the x-axis, and the downward direction as the positive direction of the y-axis. The operation unit 54 is an example of a reception unit that accepts user operations.

The term "touch" in the present specification includes all operations of bringing the input medium into contact with the display screen. That is, a tap operation for separating the touched input medium from the display screen within a predetermined time, a long touch operation for keeping the touched input medium stationary on the display screen, a slide operation for sliding the touched input medium on the display screen, and a display screen. A flick operation in which the acceleration of the input medium that slides up is above the threshold, a pinch-in operation that slides two input media that touched different positions on the display screen in a direction that brings them closer to each other, and two touches that touch different positions on the display screen. A pinch-out operation in which the input media are slid away from each other is an example of touch. Further, the input medium may be a user's finger, a stylus, or the like.

The communication unit 55 communicates with the multifunction device 10 or the service providing devices 70 and 80 through the mobile communication network or the wireless LAN. The OS 64 and the control program 65 are stored in the program storage area 62A of the storage unit 62. Specific examples of the mobile terminal 50 are not limited to smartphones, mobile phones, tablet terminals, etc., and may be PCs (abbreviations for Personal Computer) and the like. The mobile terminal 50 is an example of an information processing device.

[Service providing devices 70 and 80]
The service providing devices 70 and 80 are devices that provide services to be used by the mobile terminal 50. The service providing devices 70 and 80 exist on the Internet and provide various services in response to a request from the mobile terminal 50. The service providing device 70 is, for example, a device that provides the service of "Evernote (registered trademark of Evernote Corporation)". The service providing device 80 is, for example, a storage service such as "Dropbox (registered trademark of DropBox, Inc.)" or "Google Drive (registered trademark of Google, Inc.)" that stores data in file units, or It is a general WEB server. The service providing device 70 is an example of the first service providing device or the specific service providing device, and the service providing device 80 is an example of the second service providing device.

As shown in FIG. 3, the service providing device 70 stores the layout definition data and the subject information in association with each other. The corresponding layout definition data and subject information are uniquely identified by, for example, layout data identification information (hereinafter, referred to as “note ID”). The layout definition data stored in the service providing device 70 is not limited to one, and may be plural. Further, the service providing device 70 stores the first thumbnail image data corresponding to the layout definition data and the thumbnail image data corresponding to the content data in the image format.

The layout definition data defines the layout of the layout image. The layout definition data includes layout information indicating the arrangement of the content image and the text. The content image is an image showing content data. The text is a character string represented by the text data included in the layout definition data. That is, the layout image is generated by arranging the content image and the text according to the layout information. The subject information indicates the subject of the layout image defined by the corresponding layout definition data. The layout definition data or layout image corresponds to a so-called note, and the subject corresponds to a so-called title.

The <TEXT> tag, <FILE> tag, and <IMAGE> tag shown in FIG. 3A are examples of layout information. The <TEXT> tag is layout information indicating the arrangement of text. The <FILE> tag and the <IMAGE> tag are layout information indicating the arrangement of content images indicating the content data indicated by "src". The <IMAGE> tag corresponds to the content data in the image format, and the <FILE> tag corresponds to the content data in the non-image format. Further, the file name at the end of the URL (abbreviation of Uniform Resource Locator) indicated by "src" is an example of the identification information for identifying the content data.

The description language of the layout definition data is not particularly limited, and is, for example, HTML (abbreviation of HyperText Markup Language), XHTML (abbreviation of Extreme HyperText Language), XML (abbreviation of XML), etc. Further, the subject information may be included in the layout definition data or may be stored independently of the layout definition data.

[Operation of recording system 100]
The note printing process according to the first embodiment will be described with reference to FIGS. 4 to 11. The note printing process is a process in which the mobile terminal 50 that has acquired the layout definition data from the service providing device 70 causes the multifunction device 10 to execute a recording process of recording the content, layout image, or text indicated by the content data on paper. ..

First, the control program 65 of the mobile terminal 50 causes the display unit 53 to display the service selection screen (S11). FIG. 7A is an example of a service selection screen. The service selection screen shown in FIG. 7A includes a [first service] icon and a [second service] icon. The first service is a service provided by the service providing device 70, and the second service is a service provided by the service providing device 80. However, it goes without saying that the number of service icons included in the service selection screen is not limited to two. Then, the control program 65 waits in a state where a user operation for selecting one of the service icons included in the service selection screen can be accepted through the operation unit 54 (S12). The processes of steps S11 and S12 are examples of service selection processes.

Next, the control program 65 executes the token acquisition process in response to the operation unit 54 receiving the user operation of tapping the position of the [first service] icon (S12: Yes) (S13). The token acquisition process is a process of acquiring a token indicating that the service providing device 70 has the authority to acquire the data stored in the service providing device 70. The token is an example of authentication information. The details of the token acquisition process will be described in the second embodiment, and the following description will be continued assuming that the token has been acquired in the first embodiment. Step S13 is an example of the acquisition process.

Next, the control program 65 acquires a note list from the service providing device 70 (S14). For example, in step S14, the control program 65 receives the note ID, the subject information, and the first thumbnail image data corresponding to the layout definition data stored in the service providing device 70 from the service providing device 70 through the communication unit 55. The layout definition data, the note ID, the subject information, and the first thumbnail image data may be associated with each other and stored in the service providing device 70.

The note ID is information that identifies the layout definition data. The subject information is information indicating the title of the note (hereinafter, also referred to as "subject") defined by the layout definition data. The subject information may be text data indicating the title of the note. The first thumbnail image data is data indicating a first thumbnail image which is a thumbnail of a notebook. The first thumbnail image may be, for example, an image in which the size of a representative image, which is one of the images included in the corresponding note, is reduced. The thumbnail image may be an image in which the resolution of the original image is reduced.

More specifically, in step S14, the control program 65 transmits the list request information including the token acquired in step S13 to the service providing device 70 through the communication unit 55. Then, the control program 65 receives the subject information and the first thumbnail image data as a response to the list request information. The process of step S14 is an example of the layout data list acquisition process.

Next, the control program 65 executes the note list display process (S15). The note list display process is a process of displaying a list of layout identification images corresponding to the layout definition data on the display unit 53 in order to allow the user to select one of the layout definition data stored in the service providing device 70. The note list display process will be described in detail with reference to FIG.

First, the control program 65 determines whether or not the second thumbnail image data corresponding to one of the subject information acquired in step S14 is stored in the data storage area 62B (S31). Then, the control program 65 adopts the first thumbnail image as the layout identification image corresponding to the subject information according to the fact that the second thumbnail image data is not stored in the data storage area 62B (S31: No) (S31: No). S32).

On the other hand, the control program 65 adopts the second thumbnail image as the layout identification image corresponding to the subject information according to the fact that the second thumbnail image data is stored in the data storage area 62B (S31: Yes) (S31: Yes). S33). The second thumbnail image is an image indicated by the second thumbnail image data generated in step S19 described later. The second thumbnail image may be, for example, an image obtained by reducing the size or resolution of the layout image representing the entire corresponding note.

The control program 65 repeatedly executes the processes of steps S31 to S33 until the layout identification image corresponding to all the subject information is determined (S34: No). Then, the control program 65 causes the display unit 53 to display the note selection screen according to the determination of all the layout identification images (S34: Yes) (S35). FIG. 8 is an example of a note selection screen. On the note selection screen shown in FIG. 8, the subject indicated by the subject information and the layout identification images 111 to 113 corresponding to the subject information are listed.

Layout identification images 111 and 112 shown in FIG. 8A are examples of first thumbnail images. The first thumbnail image may show a representative image of the notebook. In other words, the first thumbnail image may be part of a notebook. On the other hand, the layout identification image 113 shown in FIG. 8B is an example of the second thumbnail image. The second thumbnail image may show the entire note. However, the first thumbnail image and the second thumbnail image corresponding to the same subject information may be the same.

Returning to FIG. 4, the control program 65 waits in a state where a user operation for selecting one of the layout identification images 111 to 113 included in the note selection screen can be accepted through the operation unit 54 (S16). The processes of steps S15 and S16 are examples of layout data selection processes. It should be noted that what is displayed in association with the layout identification images 111 to 113 on the note selection screen is not limited to the subject information, and may be text or the like included in the layout definition data. Note that the first thumbnail image data and the second thumbnail image data can be associated with not only the subject information but also the layout definition data or the note ID corresponding to the subject information. In addition, the subject information, layout definition data, and note ID are associated one-to-one. Therefore, it can be said that "corresponding to the subject information" indirectly corresponds to the layout definition data or the note ID.

Next, the control program 65 receives the layout definition data corresponding to the layout identification image according to the operation unit 54 receiving the user operation of tapping the positions of the layout identification images 111 to 113 (S16: Yes). Received from the service providing device 70 through the communication unit 55 (S17). More specifically, the control program 65 transmits the note request information including the token and the note ID to the service providing device 70 through the communication unit 55. Then, the control program 65 may receive the layout definition data specified by the note ID as a response to the note request information.

Further, in step S17, the control program 65 receives thumbnail image data corresponding to the content data in the image format defined in the layout definition data from the service providing device 70 through the communication unit 55. The control program 65 can receive the thumbnail image data together with the content data by including, for example, an option for requesting the transmission of the thumbnail image data in the HTTP request using the URL of the content data. The process of step S17 is an example of the layout data acquisition process.

Next, the control program 65 executes the note display process (S18). The note display process is a process of displaying the note display screen including the layout image on the display unit 53 by analyzing the layout definition data acquired in step S17. More specifically, the note display process is a process of displaying the layout image on the display unit 53 by arranging the text indicated by the text data and the content image indicating the content data according to the layout information. A procedure for displaying the note display screen shown in FIG. 9 on the display unit 53 according to the layout definition data of FIG. 3A will be described with reference to FIG.

First, the control program 65 arranges the text indicated by the text data on the layout image according to the layout information according to the layout information indicating the text data (S41: Yes). As an example, as shown in FIG. 9 (A), the control program 65 displays “2014/9/9 sunny” indicated by the <TEXT> tag on the first line of FIG. 3 (A) on the layout image 119. Place it on the right side of the left and right direction. As another example, the control program 65 displays "I made lunch with my family" and "It was delicious" indicated by the <TEXT> tags on the 2nd and 6th lines of FIG. 3 (A). , The layout image 119 is arranged on the left side in the left-right direction.

Further, the control program 65 arranges the thumbnail image indicated by the thumbnail image data acquired in step S17 on the layout image according to the layout information according to the layout information indicating the content data in the image format (S43: Yes). (S44). For example, the control program 65 is indicated by the <IMAGE> tag on the fifth line of FIG. 3 (A) as shown in FIG. 9 (A) [pic. As the content image corresponding to jpg], the thumbnail image 115 corresponding to the content data is arranged at the center in the left-right direction on the layout image 119.

Further, the control program 65 is stored in the data storage area 62B according to the layout information indicating the content data in the data format which is different from the image format and cannot be recorded by the compound machine 10 (S45: Yes). The non-printable image and the identification information of the content data are arranged on the layout image according to the layout information (S46). For example, the control program 65 is indicated by the <FILE> tag on the fourth line of FIG. 3 (A) as shown in FIG. 9 (A) [cooking image. As content images corresponding to [mpeg], non-printable images 116 and [cooking images. The character string of [mpeg] is arranged on the left side in the left-right direction on the layout image 119. The non-printable image 116 is an example of a non-printable image indicating that the recording process cannot be executed. The moving image format and the audio format are examples of data formats in which recording processing cannot be executed.

Further, the control program 65 corresponds to the content data according to the layout information indicating the content data in the data format different from the image format and in the data format in which the recording process by the compound machine 10 can be executed (S45: No). It is determined whether or not the thumbnail image data to be stored is stored in the data storage area 62B (S47). The thumbnail image data corresponding to the content data is generated in step S23 described later.

Then, the control program 65 receives the format image corresponding to the data format of the content data and the identification information of the content data according to the fact that the thumbnail image data is not stored in the data storage area 62B (S47: No). Is placed on the layout image according to the layout information (S49). For example, the control program 65 is indicated by the <FILE> tag on the third line of FIG. 3 (A) as shown in FIG. 9 (A) [Recipe. As the content image corresponding to [pdf], the format image 117 and [recipe. The character string of [pdf] is arranged on the left side in the left-right direction on the layout image 119.

On the other hand, the control program 65 sets the thumbnail image indicated by the thumbnail image data and the identification information of the content data according to the thumbnail image data stored in the data storage area 62B (S47: Yes). It is arranged on the layout image according to the layout information (S48). For example, the control program 65 is indicated by the <FILE> tag on the third line of FIG. 3 (A) as shown in FIG. 9 (B) [Recipe. As content images corresponding to [pdf], thumbnail images 118 and [recipe. The character string of [pdf] is arranged on the left side in the left-right direction on the layout image 120.

Then, the control program 65 repeatedly executes the processes of steps S41 to S49 until all the layout information included in the layout definition data has been analyzed (S50: No). Further, the control program 65 arranges the subject indicated by the subject information corresponding to the layout definition data at a predetermined position on the layout images 119 and 120. The subject [lunch] in Embodiment 1 is located in the upper left corner of layout images 119, 120. As a result, the note display screen shown in FIG. 9 is displayed on the display unit 53. The control program 65 may generate layout image data indicating the entire layout images 119 and 120 according to the layout definition data, and display the layout images 119 and 120 indicated by the layout image data on the display unit 53. The layout image data may be in JPEG format, for example.

The notebook display screen shown in FIG. 9A includes a layout image 119, a [layout print] icon, and a [text print] icon. The note display screen shown in FIG. 9B is different from the note display screen shown in FIG. 9A in that the layout image 120 is included in place of the layout image 119. The layout image 120 differs from the layout image 119 in that it includes a thumbnail image 118 instead of the format image 117. Then, the control program 65 waits in a state where the user operation on the note display screen can be accepted through the operation unit 54 (S20). The processes of steps S18 and S20 are examples of layout display processes.

Next, the control program 65 generates second thumbnail image data corresponding to the layout images 119 and 120 displayed on the display unit 53 in step S18 (S19). Specifically, in step S19, for example, the control program 65 generates second thumbnail image data indicating layout images 119 and 120 using the layout definition data, and uses the second thumbnail image data as note ID, subject information, and the like. Is stored in the data storage area 62B in association with.

As an example, the control program 65 creates thumbnail image data by reducing the size or resolution of the layout image data generated when displaying the note display screen, and stores the thumbnail image data in the data storage area 62B. May be good. As another example, the control program 65 may store the layout image data itself as thumbnail image data in the data storage area 62B. However, if the second thumbnail image data corresponding to the subject information is already stored in the data storage area 62B, step S19 may be omitted. The process of step S19 is an example of the thumbnail generation process.

Next, the control program 65 receives the user operation of tapping the position of the content image included in the note display screen by the operation unit 54 (S20: content image), and receives the content data corresponding to the content image. , Received from the service providing device 70 through the communication unit 55 (S21). The process of step S21 is an example of the content data acquisition process.

For example, in step S21, the control program 65 transmits the content request information including the identification information of the token and the content data to the service providing device 70 through the communication unit 55. Then, the control program 65 receives the content data identified by the identification information as a response to the content request information. However, the acquisition timing of the content data is not limited to step S21, and for example, all the content data defined in the layout definition information in step S17 may be acquired.

Next, the control program 65 causes the display unit 53 to display the preview screen of the content data acquired in step S21 (S22). FIG. 10 is an example of a preview screen. The preview screen shown in FIG. 10 includes preview images 121 and 122 showing the result when the recording process for the content data is executed according to the set value corresponding to the content data, and the [Print] icon. Then, the control program 65 waits in a state where the user operation for selecting the [print] icon can be accepted through the operation unit 54 (S26). The process of step S22 is an example of the preview display process.

In addition, for example, in step S22, the control program 65 generates preview image data in which the setting value corresponding to the data format of the content data is reflected in the image indicated by the content data, and the preview image 121 indicated by the preview image data. , 122 are included in the preview screen. Further, the control program 65 may transmit the preview image data generated when displaying the preview screen to the multifunction device 10 through the communication unit 55 in step S27 described later. The preview image data may be in JPEG format, for example.

FIG. 10A is an example of a preview screen when the operation unit 54 accepts a user operation of tapping the position of the thumbnail image 115 shown in FIG. That is, the preview image 121 is the content data acquired in step S21 [pic. The result of executing the recording process for [jpg] according to the set value corresponding to the data format "jpg" is shown. The preview image 121 according to the present embodiment is the content data [pic. The image indicated by jpg] indicates an image recorded on full-color L-size glossy paper.

FIG. 10B is an example of a preview screen when the operation unit 54 accepts a user operation of tapping the position of the format image 117 shown in FIG. 9A or the thumbnail image 118 shown in FIG. 9B. Is. That is, the preview image 122 is the content data acquired in step S21 [recipe. The result of executing the recording process for [pdf] according to the set value corresponding to the data format "pdf" is shown. The preview image 122 according to the present embodiment is the content data [recipe. The image indicated by [pdf] is reduced to 256 colors to show an image recorded on A4 size paper.

Further, when the content data is composed of a plurality of pages, the preview screen has a first preview image 122A showing the result of the recording process for the first page and a second preview showing the result of the recording process for the second page. Image 122B and the like are displayed in a list. When the content data includes page break information indicating the boundary positions of the first page and the second page, the control program 65 generates the first preview image 122A and the second preview image 122B according to the page break information. do it. On the other hand, when the page break information is not included in the content data, the control program 65 may determine the page boundary position based on the corresponding paper size, the number of lines that can be included in one page, and the like.

Further, the control program 65 generates thumbnail image data corresponding to the content data acquired in step S21 and stores it in the data storage area 62B (S23). The specific method of generating the thumbnail image data is not particularly limited, but as shown in FIG. 10B, for example, when the content data is composed of a plurality of pages, the thumbnail image data representing the content of the first page is used. It may be generated. On the other hand, the generation of thumbnail image data corresponding to the content data in the image format can be omitted. The process of step S23 is an example of the thumbnail generation process.

Next, the control program 65 transmits the recording instruction information to the multifunction device 10 through the communication unit 55 in response to the operation unit 54 receiving the user operation of tapping the position of the [print] icon (S26: Yes). (S27). The recording instruction information is information for instructing the multifunction device 10 to execute the recording process whose results are represented by the preview images 121 and 122. The process of step S27 is an example of the output process.

As an example, the recording instruction information includes print data for causing the multifunction device 10 to execute the recording process. The print data is data for executing the recording process for the content data acquired in step S21 according to the set value corresponding to the data format of the content data. The print data in this case is generated by the control program 65. As another example, the recording instruction information includes the content data acquired in step S21 or the preview image data showing the preview images 121 and 122 displayed in step S22, and the setting value corresponding to the data format of the content data. The print data in this case is generated by the control program 35 of the multifunction device 10.

Further, the control program 65 displays the note preview screen in response to the operation unit 54 accepting the user operation of tapping the position of the [layout print] icon included in the note display screen (S20: [layout print] icon). It is displayed on the display unit 53 (S24). FIG. 11A is an example of a note preview screen when the layout image 119 shown in FIG. 9A is selected. The preview screen shown in FIG. 11A includes a preview image 123 and a [print] icon.

The preview image 123 shows the result of executing the recording process for the layout image 119 according to the set value corresponding to the data format "xml" of the layout definition data. The setting value corresponding to the data format of the layout definition data is an example of the setting value for the layout image. However, among the content images included in the layout image 119, the preview image 123 does not reflect the content image corresponding to the content data in the data format for which the recording process cannot be executed. That is, the preview image 123 contains the content data [cooking video. The non-printable image 116 and the identification information corresponding to [mpeg] are not included.

In other words, among the content images included in the layout image 119, only the content image corresponding to the content data in the data format in which the recording process can be executed is reflected in the preview image 123. That is, in the preview image 123, the text indicated by the text data and the content data [recipe. Format image 117 corresponding to [pdf], identification information, and content data [pic. The thumbnail image 115 and the identification image corresponding to jpg] are included. Further, the preview image when the layout image 120 shown in FIG. 9B is selected includes the thumbnail image 118 instead of the format image 117.

Further, the control program 65 displays the text preview screen in response to the operation unit 54 accepting the user operation of tapping the position of the [text print] icon included in the note display screen (S20: [text print] icon). It is displayed on the display unit 53 (S25). FIG. 11B is an example of a text preview screen. The text preview screen shown in FIG. 11B includes a preview image 124 and a [print] icon.

The preview image 124 is an image showing the result of executing the recording process for the text extracted from the layout definition data according to the set value corresponding to the data format "txt" of the text data. The setting value corresponding to the data format of the text data is an example of the setting value for text. The text extracted from the layout definition data may be arranged according to the layout information, or may be always left-justified.

Then, the control program 65 executes the recording process whose results are represented by the preview images 123 and 124 in response to the user operation of tapping the position of the [Print] icon received by the operation unit 54 (S26: Yes). The recording instruction information for instructing is transmitted to the multifunction device 10 through the communication unit 55 (S27).

On the other hand, the control program 35 of the multifunction device 10 causes the printer unit 11 to execute the recording process indicated by the recording instruction information in response to receiving the recording instruction information from the mobile terminal 50 through the communication unit 25. As described above, the data included in the recording instruction information may be the content data itself, the preview image data indicating the preview image, or the print data. That is, as long as the multifunction device 10 can execute the recording process according to the recording instruction information, the specific example of the data included in the recording instruction information is not particularly limited.

Further, the control program 65 requests the service providing device 80 by a method corresponding to the service in response to the user operation of tapping the position of the [second service] icon received by the operation unit 54 (S12: No). Is transmitted (S28). For example, when the service providing device 80 is a general WEB server, the control program 65 may transmit a conventional HTTP request to the service providing device 80 through the communication unit 55.

[Action and effect of embodiment 1]
According to the first embodiment, the recording process for the content data is executed according to the set value corresponding to the data format of the selected content data. That is, the recording process is executed with an appropriate set value according to the data format. The control program 65 may accept a user operation for changing the set value stored in the data storage area 62B through the operation unit 54. As a result, the recording process is executed with the set value desired by the user.

Further, according to the first embodiment, the preview images 121 and 122 of the content data are displayed on the display unit 53 in response to the selection of the content data (that is, tapping the content image) on the layout images 119 and 120. To. As a result, the user can easily select the content to be recorded from the layout images 119 and 120 laid out for easy viewing, and record after confirming the result of the recording process on the preview screen. You can enter instructions.

Further, according to the first embodiment, a thumbnail image is adopted as a content image indicating the content data in the image format. As a result, the user can confirm the content of the content data in the image format on the layout image. Further, according to the first embodiment, as a content image showing content data other than the image format, a format image or a non-printable image is adopted depending on whether or not the recording process can be executed. As a result, the user can be made to recognize at a glance whether or not the recording process can be executed. Further, according to the first embodiment, a thumbnail image of content data that is not in the image format is generated when the preview image is displayed. Then, when the layout image is displayed next time, the thumbnail image is displayed instead of the format image. This makes it possible for the user to confirm the content of the content data on the layout image even for the content data other than the image format. As a result, the user can quickly select the desired content data.

The method of selecting the content image on the layout image is not limited to the tap operation, and may be, for example, a long touch operation or a double tap operation. Further, a check box corresponding to each content image is provided, and a plurality of content data may be selected at once by checking the check box. In this case, the preview images of the selected content data are displayed in a list on the preview screen. Then, by tapping the [Print] icon, the recording process for each content data is executed. As a result, the recording process for a plurality of content data included in the layout image can be collectively executed, so that the recording process for a large number of content data can be realized with a small number of operations.

Further, according to the first embodiment, not only the content data alone but also the layout image in which the content image is laid out or the text included in the content definition data can be recorded on the recording paper. Further, according to the first embodiment, the content image indicating the content data for which the recording process cannot be executed is not reflected in the preview image and is not recorded on the recording paper. However, the user may select whether or not to reflect the content image indicating the content data whose recording process cannot be executed in the preview image.

In addition, steps S23, S24, and S25 may be combined and executed in any combination. As an example, steps S23 and S24 may be executed depending on the selection of the [Layout print] icon included in the note display screen. As a result, the recording paper on which the layout image is recorded and the recording paper on which each content data is individually recorded are output from the printer unit 11. As another example, steps S23 and S25 may be executed depending on the selection of the [text print] icon included in the note display screen. As a result, the recording paper on which only the text is recorded and the recording paper on which each content data is individually recorded are output from the printer unit 11.

Further, according to the first embodiment, a thumbnail image of the note displayed on the note display screen is generated, and the thumbnail image is displayed as a layout identification image on the note selection screen displayed thereafter. As a result, the user can confirm the contents of each layout definition data. As a result, the user can quickly select the desired layout definition data.

[Embodiment 2]
Next, the note printing process according to the second embodiment will be described with reference to FIGS. 12 to 16. The configurations and processes common to those in the first embodiment are assigned the same numbers, detailed description thereof will be omitted, and differences will be mainly described. The control program 65 that executes the note printing process according to the second embodiment is different from the first embodiment in that it is started by another application by using the sharing function provided by the OS 64. Further, the note printing process according to the second embodiment is different from the first embodiment in that the acquisition source of the layout definition data can be switched depending on whether or not the token can be acquired.

First, with reference to FIG. 12, the sharing function realized by the API provided by the OS 64 of the mobile terminal 50 will be described. The sharing function is a function of causing the sharing destination application to process the data output from the sharing source application among the plurality of applications operating on the mobile terminal 50.

First, the sharing source application 41 executes the OS64 API (hereinafter, referred to as “shared API”) for realizing the sharing function (S1). The execution of the shared API requires a URI (abbreviation of Uniform Resource Identifier), a type, and an action as input arguments. The URI is information indicating the location of the shared data. The type is information indicating the data format of the shared data. The action is information indicating the type of processing (for example, viewing, editing, sending, printing, etc.) for the shared data to be executed by the sharing destination application.

On the other hand, the applications 42, 43, 44, ..., Which are candidates for the sharing destination application, declare in advance to the OS 64 the data format of the shared data that can be processed and the type of processing that can be executed. Therefore, the OS 64 identifies (S2) and identifies an application that can execute the process indicated by the action on the shared data of the data format indicated by the type from the applications 42, 43, 44, ... A list of applications is displayed on the display unit 53 (S3). Then, the OS 64 waits in a state where a user operation for selecting one of the listed applications can be accepted through the operation unit 54 (S4).

Next, the OS 64 starts the application with the URI, the type, and the action as input arguments in response to the reception of the user operation for selecting one of the listed applications by the operation unit 54 (S4). (S5). In the second embodiment, the following processing will be described assuming that the control program 65 that executes the note printing process shown in FIG. 13 is activated in step S5. The OS 64 may notify the application 41 of information indicating the application specified in step S2, and cause the application 41 to execute the processes of steps S3 to S5.

The control program 65 activated by the sharing function acquires the shared data indicated by the URI given as an input argument (S51). The URI is, for example, the file path of the text file stored in the data storage area 62B. Then, this text file may include, for example, a URL indicating the location of the layout definition data stored in the service providing device 70, and subject information corresponding to the layout definition data. Further, the URL indicating the location of the layout definition data is a note ID "001" that identifies the layout definition data on the service providing device 70, for example, "www.server1.com/shard/?id=001". Including. The process of step S51 is an example of the location information acquisition process. In addition, "www.server1.com" indicates a service providing device 70. Further, "www.server2.com2" indicates the service providing device 80.

Further, the service providing device 70 according to the second embodiment has the first layout definition data shown in FIG. 3A and the second layout shown in FIG. 15 as the layout definition data identified by the note ID "001". It stores definition data. The second layout definition data is the corresponding first layout definition data processed. The second layout definition data according to the second embodiment is described in "src" [header. jpg] and [phototer. The <IMAGE> tag in which [jpg] is set is added to the first layout definition data.

Further, in the second embodiment, in addition to the service providing devices 70 and 80, there is a service providing device 90 that provides the same service as the service providing device 70. However, the service providing devices 70 and 90 provide services independently to users in different regions, for example. Therefore, it is assumed that the token of the service providing device 70 cannot access the data stored in the service providing device 90. In addition, "www.server2.com2" is a URL indicating the service providing device 90.

Next, the control program 65 executes the token acquisition process according to the URL acquired in step S51 indicating the locations of the service providing devices 70 and 90 (S52: Yes) (S53). The details of the token acquisition process will be described with reference to FIG. The control program 65 determines that the URL indicates the location of the service providing device according to the fact that the character string indicating the service providing device is included in the URL. Specifically, the control program 65 determines that the URL indicates the location of the service providing devices 70 and 90 when the URL includes "www.server1.com" or "www.server2.com2". To do.

First, the control program 65 determines whether or not the token for accessing the service providing device indicated by the URL acquired in step S51 is stored in the data storage area 62B (S71). Next, the control program 65 determines whether or not the account information for acquiring the token is stored in the data storage area 62B according to the fact that the token is not stored in the data storage area 62B (S71: No). Judgment (S72). The processes of steps S71 and S72 are examples of determination processes.

Next, the control program 65 causes the display unit 53 to display the account information input screen according to the fact that the account information is not stored in the data storage area 62B (S72: No) (S73). FIG. 7B is an example of an account information input screen. The account information input screen shown in FIG. 7B shows the message "Enter user ID & password", a text box that accepts user ID input, a text box that accepts password input, and [Login]. ] Icon and included. Then, the control program 65 waits in a state where the user operation on the display unit 53 on which the account information input screen is displayed can be accepted through the operation unit 54 (S74). The processes of steps S73 and S74 are examples of account information acquisition processes.

Then, the control program 65 corresponds to the combination of the user ID and the password entered in the text box in response to the operation unit 54 receiving the user operation of tapping the position of the [login] icon (S74: Yes). The token is acquired from the service providing devices 70 and 90 (S75). The user ID and password are examples of account information for acquiring a token. The process of step S75 is an example of the authentication information acquisition process. The token acquisition source is not limited to the service providing devices 70 and 90 itself, and may be another device that determines the validity of the account information and issues the token corresponding to the service providing devices 70 and 90. Good. The token acquisition source is also described as a token issuing device.

The OS 64 provides an API (hereinafter, referred to as "token API") for requesting tokens of the service providing devices 70 and 90. Then, the control program 65 can transmit the account information to the token issuing device using the token API and acquire the token as a response. The token issuing device issues tokens corresponding to the area where the mobile terminal 50 exists. That is, if the mobile terminal 50 exists in the area corresponding to the service providing device 70 at the time of executing step S75, the token corresponding to the service providing device 70 is acquired and exists in the area corresponding to the service providing device 90. If so, the token corresponding to the service providing device 90 is acquired.

For example, in step S75, the control program 65 transmits the authentication request information including the user ID and password to the token issuing device through the communication unit 55. Then, the control program 65 receives the token from the token issuing device through the communication unit 55 as a response to the authentication request information. Further, in step S75, the control program 65 stores the user ID and password acquired from the user through the account information input screen and the token acquired from the token issuing device in the data storage area 62B.

The control program 65 determines whether or not the token is a token of the service providing device indicated by the URL acquired in step S51 in response to the acquisition of the token from the token issuing device in step S75 (S76: Yes). (S77). In other words, in step S77, the control program 65 has the service providing devices 70 and 90 whose location is indicated by the URL acquired in step S51, and the service providing device 70 whose data acquisition authority is granted by the token acquired in step S75. , 90 is determined to match.

Then, the control program 65 ends the token acquisition process assuming that the token acquisition is successful, depending on that the acquired token is the token of the service providing device indicated by the URL acquired in step S51 (S77: Yes). (S78). On the other hand, the control program 65 terminates the token acquisition process assuming that the token acquisition has failed, depending on that the acquired token is not the token of the service providing device indicated by the URL acquired in step S51 (S77: No). (S79). Further, the control program 65 ends the token acquisition process as if the token acquisition failed (S79) in response to the fact that the token could not be acquired in step S75 (S76: No).

Further, the control program 65 skips the processes of steps S73 and S74 according to the account information stored in the data storage area 62B (S72: Yes), and uses the stored account information in the step. The processing after S75 is executed. Further, the control program 65 skips the processes of steps S72 to S76 according to the tokens stored in the data storage area 62B (S71: Yes), and targets the stored tokens after step S77. Executes the processing of.

Returning to FIG. 13, the control program 65 acquires the first layout definition data (S55) in response to the successful acquisition of the token in step S53 (S54: Yes). For example, the control program 65 transmits the note request information including the note ID "001" and the token included in the URL acquired in step S51 to the service providing device 70 through the communication unit 55. Then, the control program 65 receives the first layout definition data identified by the note ID "001" from the service providing device 70 through the communication unit 55 as a response to the note request information. The process of step S55 is an example of the first acquisition process. The first layout definition data is an example of the first data.

Next, the control program 65 determines whether or not the subject information corresponding to the note ID "001" is described in the text file referred to in step S51 (S56). Then, the control program 65 acquires the subject information from the service providing device 70 (S57) according to the fact that the subject information is not described in the text file (S56: No). On the other hand, the control program 65 skips the process of step S57 according to the subject information described in the text file (S56: Yes).

For example, in step S57, the control program 65 transmits the subject request information including the note ID "001" to the service providing device 70 through the communication unit 55. Then, the control program 65 receives the subject information corresponding to the note ID "001" from the service providing device 70 through the communication unit 55 as a response to the subject request information. The process of step S57 is an example of the subject information acquisition process.

Next, the control program 65 causes the display unit 53 to display the preview screen shown in FIG. 16A (S58). The preview screen shown in FIG. 16A includes a layout image 125 and a [print] icon. The layout image 125 is generated by analyzing the first layout definition data acquired in step S55. Further, in the upper left corner of the layout image 125, a subject indicated by subject information is arranged. The layout image 125 is an example of the first layout image. The process of step S58 is an example of the preview display process.

Further, the control program 65 acquires the second layout definition data from the location indicated by the URL acquired in step S51 in response to the failure to acquire the token in step S53 (S54: No) (S59). The control program 65 can acquire the second layout definition data by the conventional HTTP request, for example, in step S59. Further, the acquisition source of the second layout definition data may be the service providing device 70, or may be a service providing device different from the service providing device 70. Step S59 is an example of the second acquisition process. The second layout definition data is an example of the second data.

Next, the control program 65 causes the display unit 53 to display the preview screen shown in FIG. 16 (B) (S60). The preview screen shown in FIG. 16B includes a layout image 126 and a [print] icon. The layout image 126 is generated by analyzing the second layout definition data acquired in step S59. The layout image 126 is an example of the second layout image. The process of step S60 is another example of the preview display process.

The layout image 126 is a content data [Header. Content image 127 corresponding to jpg] and content data [footer. It differs from the layout image 125 in that it includes the content image 128 corresponding to jpg]. As a result, the portion of the layout image 126 corresponding to the notebook is reduced as compared with the layout image 125. Further, the layout image 126 includes a thumbnail image 115 included in the layout image 125, a print-incompatible image 116, and an image corresponding to the format image 117. However, even if the operation unit 54 accepts the user operation of tapping each image, the process of acquiring the content data corresponding to the image is not executed. Further, the layout image 126 does not include a subject.

Then, the control program 65 waits in a state where the user operation on the preview screen can be accepted through the operation unit 54 (S61). Next, the control program 65 transmits the recording instruction information to the multifunction device 10 through the communication unit 55 in response to the user operation of tapping the position of the [print] icon received by the operation unit 54 (S61: Yes). (S62). The recording instruction information transmitted in step S62 is information for causing the multifunction device 10 to execute the recording process for the layout images 125 and 126. The process of step S62 is an example of the output process.

On the other hand, the control program 65 responds to the URL acquired in step S51 indicating the location of the service providing device 80 different from the service providing devices 70 and 90 (S52: No), without executing the token acquisition process. Is used to send a request to the service providing device 80 (S63). Specifically, the control program 65 may send an HTTP Get request using a URL. Then, the control program 65 processes the data acquired as a response to the request.

As an example, the control program 65 may display the layout image obtained by analyzing the layout definition data on the display unit 53 in response to the acquisition of the layout definition data as a response to the HTTP Get request. Further, in step S59, for example, the control program 65 may acquire the second layout definition data from the service providing device 90 by the same method as in step S63 (for example, HTTP Get request). That is, the control program 65 may execute the process in the same procedure in steps S59 and S60 and step S63.

On the other hand, the control program 65 executes the process according to the data in response to the acquisition of the data different from the layout definition data as the response of the HTTP Get request. For example, when the URL acquired in step S51 indicates the content data of the service providing device 80, the control program 65 acquires the content data indicated by the URL from the service providing device 80 in step S63 and indicates the content data. The content image to be displayed may be displayed on the display unit 53.

[Action and effect of embodiment 2]
According to the second embodiment, when the token is stored in the data storage area 62B, the first acquisition process (S55) is executed, and the layout image 125 including the content images 127 and 128 can be recorded on the sheet. .. On the other hand, when the token is not stored in the data storage area 62B, the second acquisition process (S59) is executed, and the layout image 126 including the content images 127 and 128 can be recorded on the sheet. As a result, an appropriate layout image can be recorded on the sheet depending on the presence or absence of the token.

The layout image 126 according to the second embodiment includes the content images 127 and 128 in the layout image 125. However, the processing of the layout image is not limited to the addition of the content image, and may include image processing including changing the resolution and the number of colors of the layout image. Further, what is acquired in the first acquisition process and the second acquisition process is not limited to the layout definition data. For example, the control program 65 may acquire layout image data indicating the layout image 126 in the second acquisition process.

Further, according to the token acquisition process according to the second embodiment, when the token is not stored in the data storage area 62B, the token is acquired using the account information, and the account information is not stored in the data storage area 62B. Account information is obtained from the user. This increases the possibility that the recording process for the layout image 125 that does not include the content images 127 and 128 will be executed.

In the token acquisition process according to the second embodiment, the token of the service providing device corresponding to the area where the mobile terminal 50 exists is issued. Therefore, when the token is not stored in the data storage area 62B (S71: No), the control program 65 includes the service providing devices 70 and 90 indicated by the URL acquired in step S51 and the mobile terminal 50. Steps S72 to S76 may be skipped and the process may proceed to step S79, depending on the difference between the service providing devices 70 and 90 corresponding to the region.

The location where the mobile terminal 50 exists is specified, for example, by the position information received by the mobile terminal 50 from GPS (abbreviation of Global Positioning System) or the information of the communication device to which the mobile terminal 50 is connected. As an example, when the mobile terminal 50 is Wi-Fi connected, the control program 65 may specify the position of the mobile terminal 50 by the global IP address set in the broadband router which is an example of the communication device. As another example, when the mobile terminal 50 is connected to the mobile communication network, the control program 65 may specify the position of the mobile terminal 50 based on the information of the location where the nearest base station is installed.

Further, the second acquisition process may be executed when the first acquisition process fails. The failure of the first acquisition process is, for example, a case where the first layout definition data cannot be acquired even though the note request information has been transmitted. The case where the first layout definition data cannot be acquired is, for example, the case where the first data has already been deleted from the service providing device 70, the case where the token is invalid, and the like. As a result, it is possible to prevent the recording process from becoming completely impossible when the first layout definition data cannot be acquired. Further, the second acquisition process may be executed when the subject information is not described in the text data referred to in step S51 (S56: No).

Further, the first and second embodiments can be combined. For example, as shown in FIG. 17, in the note printing process according to the second embodiment, the print control process (S71) may be executed instead of the step S58. Since the print control process corresponds to the process of steps S18 to S25 of FIG. 4, the description thereof will be omitted again.

In the multifunction device 10 and the mobile terminal 50 of each embodiment, various programs stored in the program storage areas 32A and 62A of the storage units 32 and 62 are executed by the CPUs 31 and 61, whereby the control unit of the present invention can be used. An example in which each process to be executed is realized has been described. However, the configuration of the control unit is not limited to this, and a part or all of the control unit may be realized by hardware such as an integrated circuit (also referred to as an IC (abbreviation of Integrated Circuit)).

Further, the present invention can be realized not only as a multifunction device 10 or a mobile terminal 50, but also as a program for causing the multifunction device 10 or the mobile terminal 50 to execute a process. Then, the program may be recorded and provided on a non-transitory recording medium. The non-transitory recording medium may include a storage unit mounted on a server device that can be connected to the multifunction device 10 or the mobile terminal 50 via the communication network 102, in addition to a CD-ROM, a DVD-ROM, and the like. Then, the program stored in the storage unit of the server device may be distributed as information or a signal indicating the program via a communication network 102 such as the Internet.

10 ... Multifunction device 11 ... Printer unit 25, 55 ... Communication unit 31, 61 ... CPU
35, 65 ... Control program 50 ... Mobile terminal 53 ... Display unit 54 ... Operation unit 62 ... Storage unit

Claims (12)

Display and
Memory and
An operation unit that accepts an operation of touching the display screen of the display unit,
With the communication department
It has a control unit and
The communication unit is capable of wireless communication with an image recording device and a service providing device capable of executing a recording process of recording an image on a sheet.
The storage unit is a setting value corresponding to a data format of content data that can be acquired from the service providing device, and is a content data corresponding setting value used for the recording process and layout definition data that can be acquired from the service providing device. It is a setting value corresponding to the data format of, and stores the layout definition data corresponding setting value used for the recording process, and the layout definition data is layout information indicating the arrangement of the content image indicating the content data. Includes
The control unit
A layout data acquisition process for receiving the layout definition data from the service providing device through the communication unit, and
A content image display process for displaying the operation of selecting the content image and the operation of selecting the layout definition data on the display unit in a state where the operation unit can accept the operation.
A content data acquisition process of receiving the content data indicated by the selected content image from the service providing device through the communication unit in response to the operation of selecting the content image being received by the operation unit.
Content data recording instruction screen display for displaying the content data recording instruction screen for instructing the execution of the recording process for the acquired content data according to the content data correspondence setting value corresponding to the content data on the display unit. Processing and
In response to receiving the operation instructing the execution of the recording process via the content data recording instruction screen, the content data recording instruction information instructing the execution of the recording process is transmitted to the image recording device through the communication unit. The first output process to output and
In response to the operation of selecting the layout definition data received by the operation unit, the layout definition data records the layout image in which the content image represented by the layout definition data is arranged according to the layout definition data. Layout image recording instruction screen display processing for displaying the layout image recording instruction screen for instructing execution according to the layout definition data correspondence setting value corresponding to the display unit, and
In response to the operation instructing the execution of the recording process being received by the operation unit via the layout image recording instruction screen, the layout image recording instruction information instructing the execution of the recording process is transmitted through the communication unit. A terminal device that executes a second output process that outputs to an image recording device. The control unit
In the layout data acquisition process, the identification information for identifying the content data and the thumbnail image data corresponding to the content data in the image format are further received from the service providing device through the communication unit.
In the content image display process
As the content image corresponding to the content data in the image format, the thumbnail image indicated by the thumbnail image data is included in the layout image in which the content image is arranged according to the layout information.
The terminal device according to claim 1, wherein the layout image includes a format image corresponding to the data format and the identification information for identifying the content data as the content image corresponding to the content data having a data format different from the image format. .. In the content image display process, the control unit
The format image and the identification information are included in the layout image as the content image corresponding to the content data in a data format different from the image format and in which the recording process can be executed.
A claim that includes a non-corresponding image indicating that the recording process cannot be executed and the identification information in the layout image as the content image corresponding to the content data in a data format different from the image format and in which the recording process cannot be executed. 2. The terminal device according to 2. The control unit
Further, the thumbnail generation process of generating the thumbnail image data corresponding to the acquired content data and storing it in the storage unit is further executed.
Claim 2 or claim 2 or the layout image includes the thumbnail image indicated by the thumbnail image data and the identification information as the content image corresponding to the content data for which the thumbnail image data has already been generated in the content image display process. The terminal device according to 3. The layout definition data includes text data and the layout information indicating the arrangement of text indicated by the text data.
The control unit
The terminal device according to any one of claims 1 to 4, wherein in the content image display process, the text is arranged at a position on the layout image in which the content image is arranged according to the layout information. The storage unit further stores the text setting value used for the recording process for the text.
The control unit instructs the operation unit to execute the recording process for the extracted text according to the text setting value in response to the operation instructing the extraction of the text from the layout definition data. The terminal device according to claim 5, wherein a text data recording instruction screen for displaying the text data recording instruction screen is displayed on the display unit. The first aspect of claim 1, wherein the control unit includes only the content image corresponding to the content data in a data format in which the recording process can be executed among the content images included in the layout image. Terminal equipment. The control unit
Further, the acquisition process for acquiring the authentication information indicating that the service providing device has the acquisition authority for the data stored in the service providing device is executed.
In the layout data acquisition process and the content data acquisition process, the layout definition data and the content data are received from the service providing device through the communication unit as a response of the authentication information transmitted to the service providing device through the communication unit. The terminal device according to any one of claims 1 to 7. The control unit
As the service providing device from which the layout definition data is acquired, a service selection process for accepting the selection of one of the plurality of service providing devices is further executed.
The terminal device according to claim 8, wherein the acquisition process is executed in response to the operation unit receiving the operation of selecting the specific service providing device from the plurality of the service providing devices. The control unit
The layout data list acquisition process of receiving the first thumbnail image data corresponding to the layout definition data stored in the service providing device from the service providing device through the communication unit, and
A layout in which a plurality of the layout identification images are displayed on the display unit while the operation unit can accept an operation of selecting one of the layout identification images corresponding to the layout definition data stored in the service providing device. Data selection process and
In response to the operation unit receiving the operation of touching the position of the layout identification image, the layout data acquisition that receives the layout definition data corresponding to the layout identification image from the service providing device through the communication unit. Processing and
Thumbnail generation processing that uses the layout definition data acquired in the layout data acquisition process to generate second thumbnail image data corresponding to the layout identification image displayed in the content image display process and stores it in the storage unit. And run,
In the layout data selection process
As the layout identification image corresponding to the layout definition data for which the second thumbnail image data has not been generated yet, the first thumbnail image indicated by the first thumbnail image data is displayed on the display unit.
Claims 1 to 9 for displaying the second thumbnail image represented by the second thumbnail image data on the display unit as the layout identification image corresponding to the layout definition data for which the second thumbnail image data has already been generated. The terminal device according to any one. A terminal including a display unit, a storage unit, an operation unit that accepts an operation of touching the display screen of the display unit, a first communication unit that wirelessly communicates with an image recording device and a service providing device, and a first control unit. A recording system including an apparatus, a recording unit capable of performing recording processing for recording an image on a sheet, an image recording apparatus including a second communication unit, and a second control unit.
The storage unit is a setting value corresponding to a data format of content data that can be acquired from the service providing device, and is a content data corresponding setting value used for the recording process and layout definition data that can be acquired from the service providing device. It is a setting value corresponding to the data format of, and stores the layout definition data corresponding setting value used for the recording process, and the layout definition data is layout information indicating the arrangement of the content image indicating the content data. Includes
The first control unit
A layout data acquisition process for receiving the layout definition data from the service providing device through the first communication unit, and
A content image display process for displaying the operation of selecting the content image and the operation of selecting the layout definition data on the display unit in a state where the operation unit can accept the operation.
In response to the operation of selecting the content image being received by the operation unit, the content data acquisition process of receiving the content data indicated by the selected content image from the service providing device through the first communication unit. ,
Content data recording instruction screen display for displaying the content data recording instruction screen for instructing the execution of the recording process for the acquired content data according to the content data correspondence setting value corresponding to the content data on the display unit. Processing and
In response to the operation instructing the execution of the recording process being received by the operation unit via the content data recording instruction screen, the content data recording instruction information instructing the execution of the recording process is transmitted to the first communication unit. The first output process to output to the image recording device through
In response to the operation of selecting the layout definition data received by the operation unit, the layout definition data records the layout image in which the content image represented by the layout definition data is arranged according to the layout definition data. Layout image recording instruction screen display processing for displaying the layout image recording instruction screen for instructing execution according to the layout definition data correspondence setting value corresponding to the display unit, and
In response to the operation instructing the execution of the recording process being received by the operation unit via the layout image recording instruction screen, the layout image recording instruction information instructing the execution of the recording process is transmitted to the first communication unit. The second output process of outputting to the image recording device through the image recording device is executed.
In response to receiving the content data recording instruction information or the layout image recording instruction information from the terminal device through the second communication unit, the second control unit receives the content data recording instruction information or the layout image recording instruction. A recording system that causes the recording unit to execute the recording process indicated by information. A program executed by the computer of a terminal device including a display unit, a storage unit, an operation unit that accepts an operation of touching the display screen of the display unit, a communication unit, and a computer.
The communication unit is capable of wireless communication with an image recording device and a service providing device capable of executing a recording process of recording an image on a sheet.
The storage unit is a setting value corresponding to a data format of content data that can be acquired from the service providing device, and is a content data corresponding setting value used for the recording process and layout definition data that can be acquired from the service providing device. It is a setting value corresponding to the data format of, and stores the layout definition data corresponding setting value used for the recording process, and the layout definition data is layout information indicating the arrangement of the content image indicating the content data. Includes
The program
A layout data acquisition process for receiving the layout definition data from the service providing device through the communication unit, and
A content image display process for displaying the operation of selecting the content image and the operation of selecting the layout definition data on the display unit in a state where the operation unit can accept the operation.
A content data acquisition process of receiving the content data indicated by the selected content image from the service providing device through the communication unit in response to the operation of selecting the content image being received by the operation unit.
Content data recording instruction screen display for displaying the content data recording instruction screen for instructing the execution of the recording process for the acquired content data according to the content data correspondence setting value corresponding to the content data on the display unit. Processing and
In response to the operation instructing the execution of the recording process being received by the operation unit via the content data recording instruction screen, the content data recording instruction information instructing the execution of the recording process is transmitted through the communication unit. The first output process to output to the image recording device and
In response to the operation of selecting the layout definition data received by the operation unit, the layout definition data records the layout image in which the content image represented by the layout definition data is arranged according to the layout definition data. Layout image recording instruction screen display processing for displaying the layout image recording instruction screen for instructing execution according to the layout definition data correspondence setting value corresponding to the display unit, and
In response to the operation instructing the execution of the recording process being received by the operation unit via the layout image recording instruction screen, the layout image recording instruction information instructing the execution of the recording process is transmitted through the communication unit. A program that causes the computer to execute a second output process for outputting to an image recording device.

Images