JP6642693B2 - Terminal device, recording system, and program - Google Patents

Description

The present invention relates to a technique capable of causing an image recording apparatus to execute a recording process on content data acquired from a service providing apparatus.

  2. Description of the Related Art Conventionally, there has been known a program capable of causing an image recording device to record an image represented by data acquired from a service providing device on a sheet. For example, according to a program disclosed in Patent Literature 1, when a page obtained from a web server includes a character string indicating a URL, a recording process of recording the character string indicating the URL on a sheet is performed. A recording process for recording a page indicated by a URL on a sheet can be executed by the image recording apparatus.

JP 2013-134618 A

  However, a page may include not only character string data but also data in various data formats such as image data, document data, and spreadsheet data. Further, one page may include a plurality of data having different data formats.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a technique capable of executing a recording process with an appropriate setting value according to a data format of content data included in layout definition data. To provide.

A terminal device described in this specification includes a display unit, a storage unit, an operation unit that receives a user operation of touching a display screen of the display unit, a communication unit, and a control unit. The communication unit is capable of wireless communication with an image recording apparatus and a service providing apparatus capable of executing a recording process of recording an image on a sheet. The storage unit stores first format information indicating a data format of first content data, which is content data obtainable from the service providing apparatus, in association with a first setting value used for the recording process. The second format information indicating the data format of the second content data, which is the content data, is stored in association with a second setting value used for the recording process. The first format information and the second format information have different data formats. Wherein the control unit is a content image representing the content data, the first content image representing the first content data, and provides a the content image, the arrangement of the second content image representing the second content data Layout data acquisition processing for receiving layout definition data including layout information indicating the first content image and the second content image from the service providing device through the communication unit, and performing a user operation of selecting the content image. A content image display process for displaying on the display unit in a state in which the operation unit can accept the content image; and a layout data acquisition process in response to a user operation for selecting the first content image being received by the operation unit. the first near point represented by the layout definition data The first content data indicated by the image, the a first content data acquisition process of receiving from the service providing apparatus through the communication unit, collected data format obtained by said recording process the first content data to said first content data A first preview display process for displaying, on the display unit, a first preview image representing a result of execution in accordance with the first setting value corresponding to the first setting value, and a recording process in which the result is represented by the first preview image. A first output process for transmitting, to the image recording apparatus via the communication unit, first recording instruction information for instructing execution of the recording process, in response to the operation unit receiving a user operation for instructing execution; The layout data acquisition processing is performed in response to a user operation for selecting the second content image being received by the operation unit. A second content data acquisition process of receiving the second content data indicated by the second content image indicated by the layout definition data acquired from the service providing device through the communication unit, and the acquired second content data A second preview display process for displaying, on the display unit, a second preview image representing a result of executing the recording process on the display unit in accordance with the second setting value corresponding to the data format of the second content data; In response to the operation unit receiving a user operation instructing execution of the recording process whose result is represented by the preview image, the second recording instruction information instructing execution of the recording process is transmitted through the communication unit. And a second output process to be transmitted to the image recording apparatus .

  According to the above configuration, the recording process for the selected content data is executed according to the set value corresponding to the data format of the selected content data. That is, the recording process is performed with appropriate setting values according to the data format. Note that the setting value of the recording process is, for example, a value indicating the size and orientation of the sheet, the resolution, the magnification, the number of copies, and the like.

  In addition, in response to the selection of the content data on the layout image, the result of the recording process executed on the content data under the corresponding execution condition is displayed as a preview. Thus, the user can easily select the content to be subjected to the recording process from the layout images that are easily laid out, and can input a recording instruction after confirming the result of the recording process.

  It should be noted that the “result of the recording process” in the present specification indicates an image when it is assumed that the recording process has been executed without the necessity of actually executing the recording process. That is, the preview image is an image image on the assumption that the recording process has been executed, and indicates, for example, a state in which an image represented by the content data is displayed on a sheet of a predetermined size.

  Delete

  Delete

  Delete

  Delete

  Delete

  Delete

  Delete

  Delete

  Delete

  Delete

  Delete

  Delete

  Delete

  Delete

  Delete

  Delete

  Delete

  Delete

  Delete

  Delete

  Delete

  According to the present invention, after the user confirms the result of the recording process executed in accordance with the appropriate setting according to the data format with the preview image, the recording process in which the result is represented by the preview image is executed.

FIG. 1 is a block diagram of a recording system 100 according to the embodiment. FIG. 2 is an example of format information and set values stored in the data storage area 62B. FIG. 3A is an example of layout definition data, and FIG. 3B is an example of subject information. FIG. 4 is a flowchart of the note printing process according to the first embodiment. FIG. 5 is a flowchart of the note list display processing. FIG. 6 is a flowchart of the note display processing. FIG. 7 is a display example of the display unit 53, in which (A) shows a service selection screen and (B) shows an account information input screen. FIGS. 8A and 8B are display examples of the note selection screen on the display unit 53. FIG. 8A includes the first thumbnail image 111, and FIG. 8B includes the second thumbnail image 113. 9A and 9B are display examples of the note display screen on the display unit 53. FIG. 9A includes a format image 117 and FIG. 9B includes a thumbnail image 118. FIGS. 10A and 10B are display examples of a preview screen on the display unit 53. FIG. 10A includes a preview image 121 of content data in an image format, and FIG. 10B includes a preview image 122 of content data different from the image format. FIG. 11 shows a display example of the display unit 53, in which (A) shows a note preview screen and (B) shows a text preview screen. FIG. 12 is a diagram illustrating a sharing function implemented by the sharing API. FIG. 13 is a flowchart of the note printing process according to the second embodiment. FIG. 14 is a flowchart of the token acquisition process. FIG. 15 shows another example of the layout definition data. FIGS. 16A and 16B are display examples of the display unit 53. FIG. 16A shows a preview screen including a layout image 125, and FIG. 16B shows a preview screen including a layout image 126. FIG. 17 is a flowchart of a note printing process according to a modification of the second embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings as appropriate. The embodiment described below is merely an example of the present invention, and it goes without saying that the embodiment of the present invention can be appropriately changed without changing the gist of the present invention.

  FIG. 1 is a schematic diagram of a recording system 100 according to the present embodiment. The recording system 100 shown in FIG. 1 includes a multifunction peripheral 10, a portable terminal 50, and service providing devices 70 and 80. On the other hand, the service providing apparatus 90 will be described later in a second embodiment. The MFP 10, the mobile terminal 50, and the service providing apparatuses 70 and 80 can communicate with each other via a communication network 102. A specific example of the communication network 102 is not particularly limited, but may be, for example, the Internet, a mobile communication network, a wired LAN (abbreviation of Local Area Network), a wireless LAN, or a combination thereof.

[MFP 10]
As shown in FIG. 1, the MFP 10 includes a printer unit 11, a scanner unit 12, a display unit 23, an operation unit 24, a communication unit 25, a CPU 31, a storage unit 32, a communication bus 33 Mainly equipped. The components constituting the multifunction peripheral 10 are mutually connected via a communication bus 33. The multifunction device 10 is an example of an image recording device.

[Printer 11, Scanner 12]
The printer unit 11 performs a recording process of recording an image represented by image data on a recording sheet, which is an example of a sheet. The recording method of the printer unit 11 is not particularly limited, and for example, a known method such as an ink jet method or an electrophotographic method can be adopted. The scanner unit 12 performs a scanning process of reading an image recorded on a recording sheet and generating image data. The multifunction peripheral 10 may further have a facsimile function for transmitting and receiving a facsimile, a copy function for reading an image recorded on a recording sheet and recording the image on another recording sheet, and the like.

[Display unit 23]
The display unit 23 includes a display screen for displaying various information. Although the specific configuration of the display unit 23 is not particularly limited, for example, a liquid crystal display (abbreviation for Liquid Crystal Display), an organic EL display (abbreviation for Organic Electro-Luminescence Display), or the like can be adopted.

[Operation unit 24]
The operation unit 24 receives a user operation of selecting an object displayed on the display screen of the display unit 23. Specifically, the operation unit 24 has, for example, a push button, and outputs various operation signals associated with the pushed push button to the CPU 31. Further, the operation unit 24 may include a film-shaped touch sensor superimposed on the display screen of the display unit 23. That is, the display unit 23 may be configured as a touch panel display. A well-known method such as an electrostatic capacitance method or a resistive film method can be used for the touch sensor.

  The “object” refers to an image that can be selected by the user operating the operation unit 24. As an example, the object is a character string displayed on the display unit 23, and one of the objects is highlighted by pressing a direction key of the operation unit 24, and is highlighted by pressing a determination button of the operation unit 24. The lightly displayed object may be selected. As another example, when the operation unit 24 is a touch panel, the object is an icon, a button, a link, or the like displayed on the display unit 23, and the object displayed at the touch position may be selected.

[Communication unit 25]
The communication unit 25 is an interface for communicating with an external device via the communication network 102. That is, the MFP 10 outputs various types of information to the mobile terminal 50 or the service providing devices 70 and 80 through the communication unit 25, and receives various data or various types of information from the mobile terminal 50 or the service providing devices 70 and 80 through the communication unit 25. I do. Although a specific communication procedure of the communication unit 25 is not particularly limited, for example, Wi-Fi (a registered trademark of Wi-Fi Alliance) can be adopted.

[CPU 31]
A CPU (abbreviation of Central Processing Unit) 31 controls the overall operation of the MFP 10. The CPU 31 acquires various programs described later from the storage unit 32 and executes the programs based on various information output from the operation unit 24 and various information acquired from an external device through the communication unit 25. That is, the CPU 31 and the storage unit 32 constitute an example of a control unit.

[Storage unit 32]
The storage unit 32 has a program storage area 32A and a data storage area 32B. An OS (Operating System) 34 and a control program 35 are stored in the program storage area 32A. Note that the control program 35 may be a single program or an aggregate of a plurality of programs. Data or information necessary for executing the control program 35 is stored in the data storage area 32B. As shown in FIG. 2, the data storage area 32B stores format information and setting values in association with each other.

  The format information is information indicating a data format of content data, text data, and layout definition data described below. The format information in the example of FIG. 2 includes “jpg” which is an example of a data format of image data, “pdf” which is an example of a data format different from the image format, “txt” which is an example of a data format of text data, and a layout. "Xml" which is an example of the data format of the definition data is shown. However, specific examples of the format information are not limited to these. Hereinafter, the data format of the image data is also described as an image format.

  The set value is a value indicating an execution condition when the MFP 10 executes a recording process for data in the corresponding data format. The setting values in the example of FIG. 2 include a value indicating “paper size” (for example, L size, A4, etc.), a value indicating “paper type” (for example, glossy paper, plain paper, etc.), and “color number”. And the like (for example, full color, 256 colors, monochrome, etc.). However, specific examples of the set values are not limited to these, and may include values indicating the paper orientation, resolution, magnification, number of copies, and the like.

  Note that “data” and “information” in this specification are common in that they are bits or bit strings that can be handled by a computer. “Data” refers to data that can be handled by a computer without considering the meaning of each bit. On the other hand, "information" indicates that the operation of the computer branches depending on the meaning of each bit. Further, the "instruction" is a control signal for prompting the destination device to perform the next operation, and may include information, or may have the property of information itself. There is also.

  Further, even if the format (for example, text format, binary format, flag format, etc.) is changed for each computer, "data" and "information" are the same as long as they are recognized as having the same meaning and content. Is treated as For example, information indicating “two” is stored as text information of ASCII code “0x32” in one computer, and is stored as binary information of “10” in binary notation in another computer. May be done.

  However, the distinction between “data” and “information” is not strict, and exceptional handling is allowed. For example, data may be temporarily handled as information, or information may be temporarily handled as data. Also, what is treated as data in one device may be treated as information in another device. Further, information may be extracted from data, or data may be extracted from information.

  The storage unit 32 includes, for example, RAM (abbreviation of Random Access Memory), ROM (abbreviation of Read Only Memory), EEPROM (abbreviation of Electrically Erasable Programmable Read-Only Memory), and HDD (abbreviation of Hard Disk, and HDD is an abbreviation of Hard Disk). A buffer or the like, or a combination thereof.

  Note that the storage unit 32 may be a computer-readable storage medium. The computer-readable storage medium is a non-transitory medium. Non-transitory media include recording media such as CD-ROMs and DVD-ROMs in addition to the above examples. Further, a non-transitory medium is also a tangible medium. On the other hand, an electric signal that carries a program downloaded from a server or the like on the Internet is a computer-readable signal medium that is a kind of computer-readable medium, but is a non-transitory computer-readable storage medium. Not included in the medium.

  The programs stored in the program storage area 32A are executed by the CPU 31. However, in this specification, the operation of each program may be described with the CPU 31 omitted. That is, in the following description, a description to the effect that "the program A executes the process A" may indicate that "the CPU 31 executes the process A described in the program A". The same applies to a mobile terminal 50 described later.

  The OS 34 provides an API (abbreviation for Application Programming Interface) for controlling the printer unit 11, the scanner unit 12, the display unit 23, the operation unit 24, the communication unit 25, and the like, which are hardware constituting the multifunction peripheral 10. This is a basic program. That is, each of the above-described programs controls each hardware by calling an API provided by the OS 34. However, in this specification, the operation of each program will be described with the OS 34 omitted. That is, in the following description, a description to the effect that “the program B controls the hardware C” may indicate that “the program B controls the hardware C through the API of the OS 34”. The same applies to a mobile terminal 50 described later.

[Mobile terminal 50]
As shown in FIG. 1, the mobile terminal 50 mainly includes a display unit 53, an operation unit 54, a communication unit 55, a CPU 61, a storage unit 62, and a communication bus 63. The display unit 53, the operation unit 54, the communication unit 55, the CPU 61, the storage unit 62, and the communication bus 63 included in the mobile terminal 50 include the display unit 23, the operation unit 24, the communication unit 25, the CPU 31, Since they are common to the storage unit 32 and the communication bus 33, the description will not be repeated. The CPU 61 and the storage unit 62 are examples of a control unit.

  The operation unit 54 of the mobile terminal 50 includes a film-shaped touch sensor superimposed on the display screen of the display unit 53. Then, the operation unit 54 receives a user operation of touching the display screen of the display unit 53. The operation unit 54 outputs position information indicating the position on the display screen touched by the user. The position information can be expressed, for example, as coordinates (x, y) on an xy plane where the upper left end of the display screen is the origin, the right direction is the positive direction of the x-axis, and the downward direction is the positive direction of the y-axis. The operation unit 54 is an example of a reception unit that receives a user operation.

  Note that “touch” in this specification includes all operations for bringing an input medium into contact with a display screen. That is, tap operation to separate the touched input medium from the display screen within a predetermined time, long touch operation to stop the touched input medium on the display screen, slide operation to slide the touched input medium on the display screen, display screen A flick operation in which the acceleration of an input medium that slides upward is equal to or greater than a threshold value, a pinch-in operation in which two input media that touch different positions on the display screen slide in a direction approaching each other, and two touches that touch different positions on the display screen A pinch-out operation of sliding the input media away from each other is an example of a touch. The input medium may be a user's finger, a touch pen, or the like.

  The communication unit 55 communicates with the MFP 10 or the service providing apparatuses 70 and 80 through a mobile communication network or a wireless LAN. An OS 64 and a control program 65 are stored in a program storage area 62A of the storage unit 62. Specific examples of the mobile terminal 50 are not limited to a smartphone, a mobile phone, a tablet terminal, and the like, and may be a PC (Personal Computer) or the like. The mobile terminal 50 is an example of an information processing device.

[Service providing devices 70 and 80]
The service providing devices 70 and 80 are devices that provide a service to be used by the mobile terminal 50. The service providing apparatuses 70 and 80 exist on the Internet and provide various services in response to requests from the mobile terminal 50. The service providing device 70 is, for example, a device that provides a service of “Evernote (a registered trademark of Evernote Corporation)”. The service providing apparatus 80 may be a storage service such as “Dropbox (registered trademark of DropBox, Inc.)” or “Google Drive (Google is a registered trademark of Google, Inc.)” that stores data in file units, or the like. It is a general web server. The service providing device 70 is an example of a first service providing device or a specific service providing device, and the service providing device 80 is an example of a second service providing device.

  The service providing apparatus 70 stores layout definition data and subject information in association with each other, as shown in FIG. The corresponding layout definition data and subject information are uniquely identified by, for example, layout data identification information (hereinafter, referred to as “note ID”). The layout definition data stored in the service providing device 70 is not limited to one, and may be plural. The service providing apparatus 70 stores first thumbnail image data corresponding to layout definition data and thumbnail image data corresponding to image format content data.

  The layout definition data defines a layout of a layout image. The layout definition data includes layout information indicating the arrangement of the content image and the text. The content image is an image indicating content data. The text is a character string indicated by text data included in the layout definition data. That is, the layout image is generated by arranging the content image and the text according to the layout information. The subject information indicates the subject of the layout image defined by the corresponding layout definition data. The layout definition data or the layout image corresponds to a so-called note, and the subject corresponds to a so-called title.

  A <TEXT> tag, a <FILE> tag, and an <IMAGE> tag illustrated in FIG. 3A are examples of layout information. The <TEXT> tag is layout information indicating the arrangement of text. The <FILE> tag and the <IMAGE> tag are layout information indicating an arrangement of a content image indicating content data indicated by “src”. The <IMAGE> tag corresponds to content data in an image format, and the <FILE> tag corresponds to content data in a format other than the image format. A file name at the end of a URL (abbreviation of Uniform Resource Locator) indicated by “src” is an example of identification information for identifying content data.

  The description language of the layout definition data is not particularly limited. For example, HTML (abbreviation of HyperText Markup Language), XHTML (abbreviation of Extensible HyperText Markup Language), and XML (abbreviation of Extension in XML, etc.) The subject information may be included in the layout definition data, or may be stored independently of the layout definition data.

[Operation of Recording System 100]
The notebook print processing according to the first embodiment will be described with reference to FIGS. The notebook printing process is a process in which the mobile terminal 50 that has acquired the layout definition data from the service providing apparatus 70 causes the multifunction peripheral 10 to execute a recording process of recording the content, layout image, or text indicated by the content data on a sheet. .

  First, the control program 65 of the mobile terminal 50 displays a service selection screen on the display unit 53 (S11). FIG. 7A is an example of a service selection screen. The service selection screen shown in FIG. 7A includes a [first service] icon and a [second service] icon. The first service is a service provided by the service providing apparatus 70, and the second service is a service provided by the service providing apparatus 80. However, it goes without saying that the number of service icons included in the service selection screen is not limited to two. Then, the control program 65 waits in a state where a user operation for selecting one of the service icons included in the service selection screen can be received through the operation unit 54 (S12). The processing in steps S11 and S12 is an example of a service selection processing.

  Next, in response to the operation unit 54 receiving a user operation of tapping the position of the [first service] icon (S12: Yes), the control program 65 executes a token acquisition process (S13). The token acquisition process is a process of acquiring a token indicating that the user has authority to acquire data stored in the service providing apparatus 70. A token is an example of authentication information. The details of the token acquisition processing will be described in the second embodiment. In the first embodiment, the following description will be continued on the assumption that the token has been acquired. Step S13 is an example of an acquisition process.

  Next, the control program 65 acquires a note list from the service providing device 70 (S14). The control program 65 receives the note ID, subject information, and first thumbnail image data corresponding to the layout definition data stored in the service providing device 70 from the service providing device 70 via the communication unit 55, for example, in step S14. The layout definition data, the note ID, the subject information, and the first thumbnail image data may be stored in the service providing device 70 in association with each other.

  The note ID is information for identifying layout definition data. The theme information is information indicating the title of the note defined by the layout definition data (hereinafter, also referred to as “theme”). The subject information may be text data indicating the title of the note. The first thumbnail image data is data indicating a first thumbnail image that is a thumbnail of a note. The first thumbnail image may be, for example, an image obtained by reducing the size of a representative image, which is one of the images included in the corresponding note. Note that the thumbnail image may be an image obtained by reducing the resolution of the original image.

  More specifically, in step S14, the control program 65 transmits the list request information including the token acquired in step S13 to the service providing device 70 via the communication unit 55. Then, the control program 65 receives the subject information and the first thumbnail image data as a response to the list request information. The process in step S14 is an example of a layout data list acquisition process.

  Next, the control program 65 executes a note list display process (S15). The notebook list display process is a process of displaying a list of layout identification images corresponding to the layout definition data on the display unit 53 in order to allow the user to select one of the layout definition data stored in the service providing apparatus 70. The note list display process will be described in detail with reference to FIG.

  First, the control program 65 determines whether or not the second thumbnail image data corresponding to one of the subject information acquired in step S14 is stored in the data storage area 62B (S31). Then, in response to the second thumbnail image data not being stored in the data storage area 62B (S31: No), the control program 65 adopts the first thumbnail image as the layout identification image corresponding to the subject information (S31: No). S32).

  On the other hand, in response to the second thumbnail image data being stored in the data storage area 62B (S31: Yes), the control program 65 adopts the second thumbnail image as the layout identification image corresponding to the subject information (S31: Yes). S33). The second thumbnail image is an image indicated by second thumbnail image data generated in step S19 described below. The second thumbnail image may be, for example, an image obtained by reducing the size or resolution of a layout image representing the entire corresponding note.

  The control program 65 repeatedly executes the processing of steps S31 to S33 until a layout identification image corresponding to all the subject information is determined (S34: No). Then, in response to the determination of all layout identification images (S34: Yes), the control program 65 displays a note selection screen on the display unit 53 (S35). FIG. 8 is an example of the note selection screen. On the note selection screen shown in FIG. 8, the subjects indicated by the subject information and the layout identification images 111 to 113 corresponding to the subject information are displayed in a list.

  The layout identification images 111 and 112 shown in FIG. 8A are examples of a first thumbnail image. The first thumbnail image may indicate a representative image of the note. In other words, the first thumbnail image may be a part of a note. On the other hand, the layout identification image 113 shown in FIG. 8B is an example of the second thumbnail image. The second thumbnail image may show the entire note. However, the first thumbnail image and the second thumbnail image corresponding to the same subject information may be the same.

  Returning to FIG. 4, the control program 65 waits in a state where a user operation for selecting one of the layout identification images 111 to 113 included in the note selection screen can be received through the operation unit 54 (S16). The processing in steps S15 and S16 is an example of the layout data selection processing. Note that what is displayed in association with the layout identification images 111 to 113 on the note selection screen is not limited to subject information, but may be text included in layout definition data. Note that the association with the first thumbnail image data and the second thumbnail image data is not limited to the subject information, and may be layout definition data corresponding to the subject information, a note ID, or the like. The subject information, layout definition data, and note ID are associated one-to-one. Therefore, it can be said that "corresponding to the subject information" indirectly corresponds to the layout definition data or the note ID.

  Next, in response to the operation unit 54 receiving a user operation of tapping the positions of the layout identification images 111 to 113 (S16: Yes), the control program 65 changes the layout definition data corresponding to the layout identification images to It is received from the service providing device 70 through the communication unit 55 (S17). More specifically, the control program 65 transmits the note request information including the token and the note ID to the service providing apparatus 70 via the communication unit 55. Then, the control program 65 may receive the layout definition data specified by the note ID as a response to the note request information.

  Further, the control program 65 receives the thumbnail image data corresponding to the content data in the image format defined by the layout definition data from the service providing device 70 through the communication unit 55 in step S17. The control program 65 can receive the thumbnail image data together with the content data, for example, by including an option for requesting transmission of the thumbnail image data in the HTTP request using the URL of the content data. The process in step S17 is an example of a layout data acquisition process.

  Next, the control program 65 executes a note display process (S18). The note display process is a process of displaying the note display screen including the layout image on the display unit 53 by analyzing the layout definition data acquired in step S17. More specifically, the note display process is a process of displaying a layout image on the display unit 53 by arranging text indicated by text data and a content image indicating content data according to layout information. A procedure for displaying the note display screen shown in FIG. 9 on the display unit 53 according to the layout definition data of FIG. 3A will be described with reference to FIG.

  First, in response to the layout information indicating text data (S41: Yes), the control program 65 arranges the text indicated by the text data on the layout image according to the layout information (S42). As an example, as shown in FIG. 9A, the control program 65 displays “2014/9/9 fine” indicated by the <TEXT> tag on the first line of FIG. To the right in the left-right direction. As another example, the control program 65 displays "Lunch made with family" and "Delicious food" indicated by the <TEXT> tag on the second and sixth lines in FIG. 3A. , On the layout image 119 on the left side in the left-right direction.

  In addition, in response to the layout information indicating the content data in the image format (S43: Yes), the control program 65 arranges the thumbnail image indicated by the thumbnail image data acquired in step S17 on the layout image according to the layout information. (S44). For example, as shown in FIG. 9A, the control program 65 includes the [pic. jpg], a thumbnail image 115 corresponding to the content data is arranged at the center in the left-right direction on the layout image 119.

  The control program 65 is stored in the data storage area 62B in response to the layout information indicating content data in a data format different from the image format and in which the recording process by the MFP 10 cannot be executed (S45: Yes). The non-printable image and the identification information of the content data are arranged on the layout image according to the layout information (S46). For example, as shown in FIG. 9 (A), the control program 65 includes a [cooking image. mpeg], a non-printing-compatible image 116 and [cooking video. mpeg] is arranged on the left side of the layout image 119 in the left-right direction. The non-printable image 116 is an example of a non-compliant image indicating that the recording process cannot be executed. The moving image format and the audio format are examples of data formats in which the recording process cannot be executed.

  The control program 65 responds to the content data in response to the layout information indicating content data in a data format different from the image format and in a data format in which the recording process can be executed by the multifunction peripheral 10 (S45: No). It is determined whether the thumbnail image data to be stored is stored in the data storage area 62B (S47). The thumbnail image data corresponding to the content data is generated in step S23 described later.

  Then, in response to the thumbnail image data not being stored in the data storage area 62B (S47: No), the control program 65 determines the format image corresponding to the data format of the content data and the identification information of the content data. Are arranged on the layout image according to the layout information (S49). For example, as shown in FIG. 9A, the control program 65 includes a [recipe. Tag] indicated by a <FILE> tag on the third line in FIG. 3A. pdf], the format image 117 and [recipe. pdf] is arranged on the layout image 119 on the left side in the left-right direction.

  On the other hand, in response to the thumbnail image data being stored in the data storage area 62B (S47: Yes), the control program 65 changes the thumbnail image indicated by the thumbnail image data and the identification information of the content data, It is arranged on the layout image according to the layout information (S48). For example, as shown in FIG. 9B, the control program 65 includes a [recipe. Tag] indicated by a <FILE> tag on the third line in FIG. 3A. pdf], a thumbnail image 118 and [recipe. pdf] are arranged on the layout image 120 on the left side in the left-right direction.

  Then, the control program 65 repeatedly executes the processing of steps S41 to S49 until all the layout information included in the layout definition data has been analyzed (S50: No). Further, the control program 65 arranges the subject indicated by the subject information corresponding to the layout definition data at a predetermined position on the layout images 119 and 120. The theme [lunch] in the first embodiment is arranged at the upper left corner of the layout images 119 and 120. As a result, the note display screen shown in FIG. Note that the control program 65 may generate layout image data indicating the entire layout images 119 and 120 according to the layout definition data, and cause the display unit 53 to display the layout images 119 and 120 indicated by the layout image data. The layout image data may be in JPEG format, for example.

  The notebook display screen illustrated in FIG. 9A includes a layout image 119, a [layout print] icon, and a [text print] icon. The notebook display screen shown in FIG. 9B differs from the notebook display screen shown in FIG. 9A in that a layout image 120 is included instead of the layout image 119. Layout image 120 differs from layout image 119 in that thumbnail image 118 is included in place of format image 117. Then, the control program 65 waits in a state where a user operation on the notebook display screen can be received through the operation unit 54 (S20). The processing in steps S18 and S20 is an example of the layout display processing.

  Next, the control program 65 generates second thumbnail image data corresponding to the layout images 119 and 120 displayed on the display unit 53 in step S18 (S19). Specifically, for example, in step S19, the control program 65 generates second thumbnail image data indicating the layout images 119 and 120 using the layout definition data, and stores the second thumbnail image data in a note ID, subject information, and the like. And stored in the data storage area 62B.

  As an example, the control program 65 creates thumbnail image data by reducing the size or resolution of the layout image data generated when displaying the note display screen, and stores the thumbnail image data in the data storage area 62B. Is also good. As another example, the control program 65 may store the layout image data itself as thumbnail image data in the data storage area 62B. However, when the second thumbnail image data corresponding to the subject information has already been stored in the data storage area 62B, step S19 may be omitted. The process in step S19 is an example of a thumbnail generation process.

  Next, in response to the operation unit 54 receiving a user operation of tapping the position of the content image included in the note display screen (S20: content image), the control program 65 changes the content data corresponding to the content image. , From the service providing device 70 through the communication unit 55 (S21). The process in step S21 is an example of a content data acquisition process.

  The control program 65 transmits the content request information including the token and the identification information of the content data to the service providing apparatus 70 via the communication unit 55, for example, in step S21. Then, the control program 65 receives the content data identified by the identification information as a response to the content request information. However, the acquisition timing of the content data is not limited to step S21. For example, all the content data defined by the layout definition information may be acquired in step S17.

  Next, the control program 65 causes the display unit 53 to display a preview screen of the content data acquired in step S21 (S22). FIG. 10 is an example of the preview screen. The preview screen shown in FIG. 10 includes preview images 121 and 122 representing a result when a recording process for content data is executed according to a setting value corresponding to the content data, and a [Print] icon. Then, the control program 65 waits in a state where a user operation for selecting the [Print] icon can be received through the operation unit 54 (S26). The process in step S22 is an example of a preview display process.

  The control program 65 generates, for example, in step S22, preview image data in which the setting value corresponding to the data format of the content data is reflected on the image indicated by the content data, and the preview image 121 indicated by the preview image data. , 122 are included in the preview screen. Further, the control program 65 may transmit the preview image data generated when displaying the preview screen to the multifunction peripheral 10 through the communication unit 55 in step S27 described later. The preview image data may be in JPEG format, for example.

  FIG. 10A is an example of a preview screen when the operation unit 54 receives a user operation of tapping the position of the thumbnail image 115 shown in FIG. That is, the preview image 121 includes the content data [pic. jpg] according to the set value corresponding to the data format "jpg". The preview image 121 according to the present embodiment includes content data [pic. [jpg] is an image recorded on glossy paper of L-size in full color.

  FIG. 10B shows an example of the preview screen when the operation unit 54 receives a user operation of tapping the position of the format image 117 shown in FIG. 9A or the thumbnail image 118 shown in FIG. 9B. It is. That is, the preview image 122 includes the content data [recipe. The result of executing the recording process for [pdf] according to the set value corresponding to the data format "pdf" is shown. The preview image 122 according to the present embodiment includes content data [recipe. pdf] is reduced to 256 colors and recorded on A4 size paper.

  When the content data is composed of a plurality of pages, the preview screen includes a first preview image 122A indicating the result of the recording process on the first page and a second preview image indicating the result of the recording process on the second page. A list of images 122B is displayed. If the page data indicating the boundary position between the first page and the second page is included in the content data, the control program 65 generates the first preview image 122A and the second preview image 122B according to the page break information. do it. On the other hand, when the page break information is not included in the content data, the control program 65 may determine the page boundary position based on the corresponding sheet size or the number of lines that can be included in one page.

  Further, the control program 65 generates thumbnail image data corresponding to the content data acquired in step S21 and stores the thumbnail image data in the data storage area 62B (S23). Although a specific method of generating the thumbnail image data is not particularly limited, for example, as shown in FIG. 10B, when the content data is composed of a plurality of pages, the thumbnail image data representing the contents of the first page is generated. May be generated. On the other hand, generation of thumbnail image data corresponding to image format content data can be omitted. The process in step S23 is an example of a thumbnail generation process.

  Next, the control program 65 transmits the recording instruction information to the MFP 10 through the communication unit 55 in response to the operation unit 54 receiving a user operation of tapping the position of the [Print] icon (S26: Yes). (S27). The recording instruction information is information for instructing the MFP 10 to execute a recording process in which the results are represented by the preview images 121 and 122. The process in step S27 is an example of an output process.

  As an example, the recording instruction information includes print data for causing the MFP 10 to execute a recording process. The print data is data for executing a recording process on the content data acquired in step S21 according to a setting value corresponding to the data format of the content data. The print data in this case is generated by the control program 65. As another example, the recording instruction information includes the content data acquired in step S21 or the preview image data indicating the preview images 121 and 122 displayed in step S22, and a setting value corresponding to the data format of the content data. The print data in this case is generated by the control program 35 of the MFP 10.

  The control program 65 changes the note preview screen in response to the operation unit 54 receiving a user operation of tapping the position of the [layout print] icon included in the note display screen (S20: [layout print] icon). It is displayed on the display unit 53 (S24). FIG. 11A is an example of the note preview screen when the layout image 119 shown in FIG. 9A is selected. The preview screen shown in FIG. 11A includes a preview image 123 and a [Print] icon.

  The preview image 123 shows the result of executing the recording process on the layout image 119 according to the setting value corresponding to the data format “xml” of the layout definition data. The setting value corresponding to the data format of the layout definition data is an example of a setting value for a layout image. However, the preview image 123 does not reflect the content image corresponding to the content data of the data format in which the recording process cannot be executed among the content images included in the layout image 119. That is, content data [cooking video. mpeg] is not included.

  In other words, in the preview image 123, only the content image corresponding to the content data in the data format in which the recording process can be executed among the content images included in the layout image 119 is reflected. That is, the preview image 123 includes text represented by text data and content data [recipe. pdf] and the content image [pic. jpg] and the identification image. Also, the preview image when layout image 120 shown in FIG. 9B is selected includes thumbnail image 118 instead of format image 117.

  Further, in response to the operation unit 54 receiving a user operation of tapping the position of the [text print] icon included in the notebook display screen (S20: [text print] icon), the control program 65 displays the text preview screen. It is displayed on the display unit 53 (S25). FIG. 11B is an example of a text preview screen. The text preview screen shown in FIG. 11B includes a preview image 124 and a [Print] icon.

  The preview image 124 is an image indicating a result of executing a recording process on the text extracted from the layout definition data in accordance with a setting value corresponding to the data format “txt” of the text data. The setting value corresponding to the data format of the text data is an example of a text setting value. The text extracted from the layout definition data may be arranged according to the layout information, or may be always arranged left-aligned.

  Then, in response to the operation unit 54 receiving the user operation of tapping the position of the [Print] icon (S26: Yes), the control program 65 executes the recording process in which the results are displayed in the preview images 123 and 124. Is transmitted to the MFP 10 through the communication unit 55 (S27).

  On the other hand, in response to receiving the recording instruction information from the portable terminal 50 through the communication unit 25, the control program 35 of the MFP 10 causes the printer unit 11 to execute the recording process indicated by the recording instruction information. As described above, the data included in the recording instruction information may be the content data itself, may be preview image data indicating a preview image, or may be print data. That is, as long as the MFP 10 can execute the recording process according to the recording instruction information, a specific example of the data included in the recording instruction information is not particularly limited.

  Further, in response to the operation unit 54 receiving a user operation of tapping the position of the [second service] icon (S12: No), the control program 65 sends a request to the service providing apparatus 80 by a method corresponding to the service. Is transmitted (S28). For example, when the service providing apparatus 80 is a general WEB server, the control program 65 may transmit a conventional HTTP request to the service providing apparatus 80 via the communication unit 55.

[Operation and Effect of First Embodiment]
According to the first embodiment, the recording process for the selected content data is executed according to the setting value corresponding to the data format of the content data. That is, the recording process is performed with appropriate setting values according to the data format. The control program 65 may receive a user operation for changing the setting value stored in the data storage area 62B through the operation unit 54. As a result, the recording process is executed with the setting values desired by the user.

  According to the first embodiment, in response to selection of content data on layout images 119 and 120 (that is, tapping of a content image), preview images 121 and 122 of the content data are displayed on display unit 53. You. This allows the user to easily select the content to be subjected to the recording process from the layout images 119 and 120 laid out for easy viewing, and to confirm the result of the recording process on the preview screen before recording. You can enter instructions.

  According to the first embodiment, a thumbnail image is adopted as a content image indicating content data in an image format. This allows the user to confirm the content of the content data in the image format on the layout image. Further, according to the first embodiment, a format image or a non-printable image is adopted as a content image indicating content data in a format other than the image format according to whether or not the recording process can be performed. This allows the user to recognize at a glance whether or not the recording process can be executed. Furthermore, according to the first embodiment, a thumbnail image of content data that is not in an image format is generated when a preview image is displayed. Then, when the layout image is displayed next time, the thumbnail image is displayed instead of the format image. This allows the user to confirm the content of the content data on the layout image even for content data other than the image format. As a result, desired content data can be promptly selected by the user.

  The method of selecting a content image on a layout image is not limited to a tap operation, and may be, for example, a long touch operation or a double tap operation. Further, a check box corresponding to each content image is provided, and a plurality of pieces of content data may be collectively selected by checking the check box. In this case, a preview image of each selected content data is displayed in a list on the preview screen. Then, when the [Print] icon is tapped, recording processing for each piece of content data is executed. Thus, the recording process for a plurality of content data included in the layout image can be collectively executed, so that the recording process for a large number of content data can be realized with a small number of operations.

  Further, according to the first embodiment, not only the content data alone, but also a layout image in which the content image is laid out or a text included in the content definition data can be recorded on a recording sheet. Further, according to the first embodiment, the content image indicating the content data for which the recording process cannot be executed is not reflected on the preview image and is not recorded on the recording paper. However, the user may select whether or not to reflect the content image indicating the content data for which the recording process cannot be executed on the preview image.

  Steps S23, S24, and S25 may be performed in any combination. As an example, steps S23 and S24 may be executed in response to selection of the [layout print] icon included in the note display screen. Thereby, the recording sheet on which the layout image is recorded and the recording sheet on which each content data is individually recorded are output from the printer unit 11. As another example, steps S23 and S25 may be executed in response to the selection of the [text print] icon included in the note display screen. As a result, recording paper on which only text is recorded and recording paper on which each content data is individually recorded are output from the printer unit 11.

  According to the first embodiment, a thumbnail image of a note displayed on the note display screen is generated, and the thumbnail image is displayed as a layout identification image on a note selection screen displayed thereafter. This allows the user to confirm the contents of each layout definition data. As a result, the user can promptly select desired layout definition data.

[Embodiment 2]
Next, a notebook print process according to the second embodiment will be described with reference to FIGS. The same components and processes as those in the first embodiment are denoted by the same reference numerals, detailed description thereof will be omitted, and differences will be mainly described. The control program 65 for executing the note print processing according to the second embodiment is different from the first embodiment in that the control program 65 is started by another application using the sharing function provided by the OS 64. Further, the note printing process according to the second embodiment is different from the first embodiment in that the acquisition source of the layout definition data is switched according to whether a token can be acquired.

  First, a sharing function realized by an API provided by the OS 64 of the mobile terminal 50 will be described with reference to FIG. The sharing function is a function of causing a sharing destination application to process data output from a sharing source application among a plurality of applications operating on the mobile terminal 50.

  First, the sharing source application 41 executes an API of the OS 64 for realizing the sharing function (hereinafter, referred to as “shared API”) (S1). To execute the shared API, a URI (abbreviation of Uniform Resource Identifier), a type, and an action are obtained as input arguments. The URI is information indicating the location of the shared data. The type is information indicating the data format of the shared data. The action is information indicating the type of processing (for example, browsing, editing, transmission, printing, etc.) on the shared data to be executed by the application at the sharing destination.

  On the other hand, the applications 42, 43, 44,..., Which are candidates for the application of the sharing destination, declare the data format of the processable shared data and the type of executable process to the OS 64 in advance. Then, the OS 64 specifies (S2) an application that can execute the process indicated by the action on the shared data in the data format indicated by the type, from the applications 42, 43, 44,... A list of applications is displayed on the display unit 53 (S3). Then, the OS 64 waits in a state where a user operation for selecting one of the applications displayed in the list can be received through the operation unit 54 (S4).

  Next, in response to the operation unit 54 receiving a user operation for selecting one of the applications displayed in the list (S4), the OS 64 activates the application with the URI, type, and action as input arguments. (S5). In the second embodiment, the following processing will be described on the assumption that the control program 65 for executing the note printing processing shown in FIG. 13 has been started in step S5. Note that the OS 64 may notify the application 41 of information indicating the application specified in step S2, and cause the application 41 to execute the processes in steps S3 to S5.

  The control program 65 started by the sharing function acquires the shared data indicated by the URI given as an input argument (S51). The URI is, for example, a file path of a text file stored in the data storage area 62B. The text file may include, for example, a URL indicating the location of the layout definition data stored in the service providing apparatus 70 and subject information corresponding to the layout definition data. Further, the URL indicating the location of the layout definition data includes a note ID “001” identifying the layout definition data on the service providing apparatus 70, for example, “www.server1.com/shared/?id=001”. Including. The process in step S51 is an example of a location information acquisition process. Note that “www.server1.com” indicates the service providing apparatus 70. “Www.server2.com2” indicates the service providing apparatus 80.

  Further, the service providing apparatus 70 according to the second embodiment includes the first layout definition data shown in FIG. 3A and the second layout shown in FIG. 15 as layout definition data identified by the note ID “001”. And definition data. The second layout definition data is obtained by processing the corresponding first layout definition data. The second layout definition data according to the second embodiment includes “header. jpg] and [footer. <img> in which <img> is set to the first layout definition data.

  Further, in the second embodiment, in addition to the service providing devices 70 and 80, there is a service providing device 90 that provides the same service as the service providing device 70. However, the service providing apparatuses 70 and 90 independently provide services to users in different areas, for example. Therefore, it is assumed that the token stored in the service providing apparatus 70 cannot access the data stored in the service providing apparatus 90. Note that “www.server2.com2” is a URL indicating the service providing apparatus 90.

  Next, the control program 65 executes token acquisition processing in response to the URL acquired in step S51 indicating the location of the service providing apparatuses 70 and 90 (S52: Yes) (S53). The details of the token acquisition process will be described with reference to FIG. In addition, the control program 65 determines that the URL is a URL indicating the location of the service providing apparatus in response to the fact that the character string indicating the service providing apparatus is included in the URL. Specifically, when the URL includes “www.server1.com” or “www.server2.com2”, the control program 65 determines that the URL indicates the location of the service providing apparatuses 70 and 90. I do.

  First, the control program 65 determines whether or not a token for accessing the service providing device indicated by the URL acquired in step S51 is stored in the data storage area 62B (S71). Next, in response to the fact that the token is not stored in the data storage area 62B (S71: No), the control program 65 determines whether or not account information for acquiring the token is stored in the data storage area 62B. A determination is made (S72). The processing of steps S71 and S72 is an example of the determination processing.

  Next, in response to the fact that the account information is not stored in the data storage area 62B (S72: No), the control program 65 displays an account information input screen on the display unit 53 (S73). FIG. 7B is an example of an account information input screen. The account information input screen shown in FIG. 7 (B) includes a message “Please enter your user ID and password.”, A text box for receiving the input of the user ID, a text box for receiving the input of the password, and ] Icon. Then, the control program 65 waits in a state where a user operation on the display unit 53 on which the account information input screen is displayed can be received through the operation unit 54 (S74). The processing of steps S73 and S74 is an example of account information acquisition processing.

  Then, in response to the operation unit 54 receiving a user operation of tapping the position of the [login] icon (S74: Yes), the control program 65 corresponds to the combination of the user ID and the password input in the text box. The token is obtained from the service providing devices 70 and 90 (S75). The user ID and the password are examples of account information for acquiring a token. The process in step S75 is an example of an authentication information acquisition process. The source of the token is not limited to the service providing apparatuses 70 and 90 itself, but may be another apparatus that determines the validity of the account information and issues a token corresponding to the service providing apparatuses 70 and 90. Good. The source of the token is also described as a token issuing device.

  Note that the OS 64 provides an API (hereinafter, referred to as a “token API”) for requesting a token of the service providing apparatuses 70 and 90. Then, the control program 65 can transmit the account information to the token issuing device using the token API, and acquire the token as a response. Note that the token issuing device issues a token corresponding to the area where the mobile terminal 50 is located. That is, when the mobile terminal 50 is located in the area corresponding to the service providing apparatus 70 at the time of execution of step S75, a token corresponding to the service providing apparatus 70 is acquired, and is present in the area corresponding to the service providing apparatus 90. If so, a token corresponding to the service providing apparatus 90 is obtained.

  The control program 65 transmits the authentication request information including the user ID and the password to the token issuing device via the communication unit 55 in step S75, for example. Then, the control program 65 receives the token as a response to the authentication request information from the token issuing device through the communication unit 55. Further, in step S75, the control program 65 causes the data storage area 62B to store the user ID and password obtained from the user through the account information input screen and the token obtained from the token issuing device.

  In response to obtaining the token from the token issuing device in step S75 (S76: Yes), the control program 65 determines whether the token is a token of the service providing device indicated by the URL obtained in step S51. (S77). In other words, the control program 65 determines in step S77 that the service providing apparatuses 70 and 90 whose location is indicated by the URL acquired in step S51 and the service providing apparatus 70 in which the token acquired in step S75 is authorized to acquire data. , 90 are determined.

  Then, in response to the acquired token being the token of the service providing apparatus indicated by the URL acquired in step S51 (S77: Yes), the control program 65 ends the token acquisition process as having succeeded in acquiring the token. (S78). On the other hand, in response to the fact that the acquired token is not the token of the service providing apparatus indicated by the URL acquired in step S51 (S77: No), the control program 65 ends the token acquisition process as a token acquisition failure. (S79). In addition, in response to the failure in step S75 to acquire the token in step S75 (S76: No), the control program 65 ends the token acquisition process assuming that the token acquisition has failed (S79).

  Further, in response to the account information being stored in the data storage area 62B (S72: Yes), the control program 65 skips the processing of steps S73 and S74, and performs the step using the stored account information. The processing after S75 is executed. Further, in response to the token being stored in the data storage area 62B (S71: Yes), the control program 65 skips the processing of steps S72 to S76 and executes the processing from step S77 onward for the stored token. Execute the processing of

  Returning to FIG. 13, the control program 65 acquires the first layout definition data in response to the successful acquisition of the token in step S53 (S54: Yes) (S55). For example, the control program 65 transmits note request information including the note ID “001” and the token included in the URL acquired in step S51 to the service providing apparatus 70 via the communication unit 55. Then, the control program 65 receives the first layout definition data identified by the note ID “001” from the service providing device 70 via the communication unit 55 as a response to the note request information. The process in step S55 is an example of a first acquisition process. The first layout definition data is an example of first data.

  Next, the control program 65 determines whether or not the subject information corresponding to the note ID “001” is described in the text file referred to in step S51 (S56). Then, when the subject information is not described in the text file (S56: No), the control program 65 acquires the subject information from the service providing device 70 (S57). On the other hand, the control program 65 skips the processing of step S57 in response to the subject information being described in the text file (S56: Yes).

  The control program 65 transmits the subject request information including the note ID “001” to the service providing apparatus 70 via the communication unit 55 in step S57, for example. Then, the control program 65 receives the subject information corresponding to the note ID “001” from the service providing device 70 via the communication unit 55 as a response to the subject request information. The process in step S57 is an example of a subject information acquisition process.

  Next, the control program 65 causes the display unit 53 to display the preview screen shown in FIG. 16A (S58). The preview screen shown in FIG. 16A includes a layout image 125 and a [Print] icon. The layout image 125 is generated by analyzing the first layout definition data obtained in step S55. In the upper left corner of the layout image 125, a subject indicated by subject information is arranged. The layout image 125 is an example of a first layout image. The process in step S58 is an example of a preview display process.

  In addition, in response to the failure in acquiring the token in step S53 (S54: No), the control program 65 acquires the second layout definition data from the location indicated by the URL acquired in step S51 (S59). For example, in step S59, the control program 65 can acquire the second layout definition data by a conventional HTTP request. The source of the second layout definition data may be the service providing device 70 or a service providing device different from the service providing device 70. Step S59 is an example of a second acquisition process. The second layout definition data is an example of the second data.

  Next, the control program 65 causes the display unit 53 to display the preview screen shown in FIG. 16B (S60). The preview screen shown in FIG. 16B includes a layout image 126 and a [Print] icon. The layout image 126 is generated by analyzing the second layout definition data obtained in step S59. The layout image 126 is an example of a second layout image. The process of step S60 is another example of the preview display process.

  The layout image 126 includes the content data [Header. jpg] and the content data [footer.jpg]. jpg] in that the content image 128 and the layout image 125 are included. As a result, in the layout image 126, a portion corresponding to the note is reduced in comparison with the layout image 125. Further, the layout image 126 includes images corresponding to the thumbnail image 115, the non-printable image 116, and the format image 117 included in the layout image 125. However, even if the operation unit 54 receives a user operation of tapping each image, the process of acquiring the content data corresponding to the image is not executed. Further, the layout image 126 does not include a subject.

  Then, the control program 65 waits in a state where a user operation on the preview screen can be received through the operation unit 54 (S61). Next, the control program 65 transmits the recording instruction information to the MFP 10 through the communication unit 55 in response to the operation unit 54 receiving a user operation of tapping the position of the [Print] icon (S61: Yes). (S62). The recording instruction information transmitted in step S62 is information for causing the multifunction peripheral 10 to execute recording processing for the layout images 125 and 126. The process in step S62 is an example of an output process.

  On the other hand, when the URL acquired in step S51 indicates the location of the service providing apparatus 80 different from the service providing apparatuses 70 and 90 (S52: No), the control program 65 executes the URL without executing the token acquisition processing. The request is transmitted to the service providing apparatus 80 by using (S63). Specifically, the control program 65 may transmit an HTTP Get request using a URL. Then, the control program 65 performs processing on the data obtained as a response to the request.

  As an example, in response to acquiring the layout definition data as a response to the HTTP Get request, the control program 65 may cause the display unit 53 to display a layout image obtained by analyzing the layout definition data. The control program 65 may also acquire the second layout definition data from the service providing apparatus 90 in step S59, for example, in the same manner as in step S63 (for example, an HTTP Get request). That is, the control program 65 may execute the processing in steps S59 and S60 and step S63 in a similar procedure.

  On the other hand, when the control program 65 obtains data different from the layout definition data as a response to the HTTP Get request, the control program 65 executes a process corresponding to the data. For example, when the URL acquired in step S51 indicates the content data of the service providing apparatus 80, the control program 65 acquires the content data indicated by the URL from the service providing apparatus 80 in step S63, and indicates the content data in the content data. The content image to be displayed may be displayed on the display unit 53.

[Operation and Effect of Second Embodiment]
According to the second embodiment, when the token is stored in the data storage area 62B, the first acquisition processing (S55) is executed, and the layout image 125 that does not include the content images 127 and 128 can be recorded on the sheet. . On the other hand, when the token is not stored in the data storage area 62B, the second acquisition processing (S59) is executed, and the layout image 126 including the content images 127 and 128 can be recorded on the sheet. As a result, an appropriate layout image can be recorded on the sheet according to the presence or absence of the token.

  Note that the layout image 126 according to the second embodiment includes the layout image 125 and the content images 127 and 128. However, the processing of the layout image is not limited to the addition of the content image, and may include image processing including changing the resolution and the number of colors of the layout image. Also, what is acquired in the first acquisition processing and the second acquisition processing is not limited to the layout definition data. For example, the control program 65 may acquire layout image data indicating the layout image 126 in the second acquisition processing.

  Further, according to the token acquisition processing according to the second embodiment, when the token is not stored in the data storage area 62B, the token is acquired using the account information, and when the account information is not stored in the data storage area 62B. First, account information is obtained from the user. This increases the possibility that the recording process is performed on the layout image 125 that does not include the content images 127 and 128.

  In the token acquisition process according to the second embodiment, a token of the service providing device corresponding to the area where the mobile terminal 50 exists is issued. Therefore, when the token is not stored in the data storage area 62B (S71: No), the control program 65 determines that the service providing apparatuses 70 and 90 indicated by the URL acquired in step S51 and the mobile terminal 50 exist. Depending on the difference between the service providing devices 70 and 90 corresponding to the region where the user is located, steps S72 to S76 may be skipped and the process may proceed to step S79.

  The location where the mobile terminal 50 exists is specified by, for example, position information received by the mobile terminal 50 from GPS (abbreviation of Global Positioning System) or information of a communication device to which the mobile terminal 50 is connected. As an example, when the mobile terminal 50 is Wi-Fi connected, the control program 65 may specify the position of the mobile terminal 50 by a global IP address set in a broadband router as an example of a communication device. As another example, when the mobile terminal 50 is connected to a mobile communication network, the control program 65 may specify the position of the mobile terminal 50 based on information on the location where the nearest base station is installed.

  Further, the second acquisition processing may be executed when the first acquisition processing has failed. The failure of the first acquisition processing is, for example, a case where the first layout definition data cannot be acquired despite the transmission of the note request information. The case where the first layout definition data cannot be acquired is, for example, a case where the first data has already been deleted from the service providing apparatus 70, a case where the token is invalid, and the like. As a result, when the first layout definition data cannot be obtained, it is possible to prevent the recording process from being disabled at all. Further, the second acquisition process may be executed when the subject information is not described in the text data referred to in step S51 (S56: No).

  Embodiments 1 and 2 can be combined. For example, as shown in FIG. 17, in the note print processing according to the second embodiment, a print control processing (S71) may be executed instead of step S58. The print control processing corresponds to the processing of steps S18 to S25 in FIG.

  In the multifunction peripheral 10 and the mobile terminal 50 of each embodiment, the various programs stored in the program storage areas 32A and 62A of the storage units 32 and 62 are executed by the CPUs 31 and 61, so that the control unit of the present invention is implemented. The example in which each processing to be executed is realized has been described. However, the configuration of the control unit is not limited to this, and a part or all of the control unit may be implemented by hardware such as an integrated circuit (also referred to as an integrated circuit (IC)).

  Furthermore, the present invention can be realized not only as the multifunction device 10 or the portable terminal 50 but also as a program for causing the multifunction device 10 or the mobile terminal 50 to execute a process. Then, the program may be provided by being recorded on a non-transitory recording medium. The non-transitory recording medium may include a storage unit mounted on a server device that can be connected to the MFP 10 or the portable terminal 50 via the communication network 102, in addition to a CD-ROM, a DVD-ROM, or the like. Then, the program stored in the storage unit of the server device may be distributed as information or a signal indicating the program via the communication network 102 such as the Internet.

10 Multifunction machine 11 Printer units 25 and 55 Communication units 31 and 61 CPU
35, 65 control program 50 mobile terminal 53 display unit 54 operation unit 62 storage unit

Claims (13)

A display unit,
A storage unit,
An operation unit that receives a user operation of touching a display screen of the display unit,
A communication unit;
And a control unit,
The communication unit is capable of wireless communication with an image recording apparatus and a service providing apparatus capable of executing a recording process of recording an image on a sheet,
The storage unit stores first format information indicating a data format of first content data, which is content data obtainable from the service providing apparatus, in association with a first setting value used for the recording process. And the second format information indicating the data format of the second content data, which is the content data, and the second setting value used for the recording process are stored in association with each other, and the first format information and the second Two-format information is a different data format,
The control unit includes:
A content image indicating the content data, a first content image indicating the first content data, and a layout information indicating an arrangement of the second content image indicating the second content data. Including layout definition data, a layout data acquisition process of receiving from the service providing device through the communication unit,
A content image display process of displaying the first content image and the second content image on the display unit in a state where a user operation for selecting the content image can be received by the operation unit;
In response to a user operation for selecting the first content image being received by the operation unit, the first content data indicated by the first content image indicated by the layout definition data acquired in the layout data acquisition process is displayed. A first content data acquisition process received from the service providing device through the communication unit;
Displaying a first preview image on the display unit, the first preview image representing a result of executing the recording processing on the acquired first content data in accordance with the first setting value corresponding to the data format of the first content data; Preview display processing,
In response to the operation unit receiving a user operation instructing execution of the recording process in which a result is displayed in the first preview image, the first recording instruction information instructing execution of the recording process is transmitted to the communication unit. A first output process for transmitting to the image recording device through the unit,
In response to a user operation for selecting the second content image being received by the operation unit, the second content data indicated by the layout definition data acquired in the layout data acquisition process is displayed. A second content data acquisition process received from the service providing device through the communication unit;
Displaying a second preview image on the display unit, the second preview image representing a result of executing the recording processing on the acquired second content data in accordance with the second setting value corresponding to the data format of the second content data. Preview display processing,
In response to the operation unit receiving a user operation instructing execution of the recording process in which the result is represented by the second preview image, the second recording instruction information instructing execution of the recording process is transmitted to the communication unit. And a second output process of transmitting the image data to the image recording device through a unit. The storage unit further stores a set value for a layout image used for the recording process on a layout image in which the content image is arranged according to the layout information ,
In response to the operation unit receiving a user operation for selecting the layout image, the control unit displays a preview image representing a result of executing the recording process on the layout image according to the layout image setting value. In response to the operation unit receiving a user operation instructing execution of the recording process, the information is displayed on the display unit, and information instructing execution of the recording process is transmitted to the image recording apparatus through the communication unit. The terminal device according to claim 1. The control unit includes:
In the layout data acquisition process, identification information for identifying the content data, and thumbnail image data corresponding to the content data in an image format, further received from the service providing device through the communication unit,
In the content image display processing,
As the content image corresponding to the content data in an image format, a thumbnail image indicated by the thumbnail image data is included in a layout image in which the content image is arranged according to the layout information ,
3. The layout image according to claim 1, wherein the layout image includes a format image corresponding to the data format and the identification information for identifying the content data, as the content image corresponding to the content data having a data format different from an image format. Terminal device. In the content image display process, the control unit includes:
As the content image corresponding to the content data in a data format that is different from an image format and in which the recording process can be performed, the format image and the identification information are included in the layout image,
A non-compliant image indicating that the recording process cannot be executed and the identification information are included in the layout image as the content image corresponding to the content data in a data format different from an image format and in which the recording process cannot be executed. 4. The terminal device according to 3. The control unit includes:
Further performing a thumbnail generation process of generating thumbnail image data corresponding to the acquired content data and storing the thumbnail image data in the storage unit;
4. The content image display process, wherein the thumbnail image and the identification information indicated by the thumbnail image data are included in the layout image as the content image corresponding to the content data for which the thumbnail image data has already been generated. 5. The terminal device according to 4. The layout definition data includes text data and the layout information indicating an arrangement of text indicated by the text data,
The control unit includes:
The terminal device according to claim 1, wherein in the content image display processing, the text is arranged at a position on a layout image where the content image is arranged according to the layout information . The storage unit further stores a text setting value used in the recording process for the text,
The control unit is configured to execute a recording process on the extracted text according to the text setting value in response to the operation unit receiving a user operation instructing extraction of the text from the layout definition data. The terminal device according to claim 6, wherein a preview image representing a result is displayed on the display unit.   3. The preview image according to claim 2, wherein the control unit includes only the content image corresponding to the content data in a data format in which the recording process is executable, among the content images included in the layout image. Terminal device. The control unit includes:
Further performing an acquisition process of acquiring authentication information indicating that the user has the authority to acquire data stored in the service providing device,
In the layout data obtaining process, the first content data obtaining process, and the second content data obtaining process, the service providing device is transmitted through the communication unit as a response to the authentication information transmitted to the service providing device through the communication unit. The terminal device according to claim 1, wherein the terminal device receives the layout definition data, the first content data, and the second content data from a terminal. The control unit includes:
Further performing a service selection process for allowing a user to select one of the plurality of service providing devices as the service providing device from which the layout definition data is obtained;
The terminal device according to claim 9, wherein the acquisition process is executed in response to the operation unit receiving a user operation for selecting a specific service providing device from among the plurality of service providing devices. The control unit includes:
A layout data list obtaining process of receiving first thumbnail image data corresponding to the layout definition data stored in the service providing device from the service providing device through the communication unit;
Displaying the plurality of layout identification images on the display unit in a state where the operation unit can receive a user operation for selecting one of the layout identification images corresponding to the layout definition data stored in the service providing device. Layout data selection processing,
In response to the operation unit receiving a user operation of touching the position of the layout identification image, the layout data receiving the layout definition data corresponding to the layout identification image from the service providing device through the communication unit. Acquisition processing,
Thumbnail generation processing for generating second thumbnail image data corresponding to the layout identification image displayed in the content image display processing using the layout definition data acquired in the layout data acquisition processing and storing the second thumbnail image data in the storage unit And run,
In the layout data selection process,
Causing the display unit to display a first thumbnail image indicated by the first thumbnail image data as the layout identification image corresponding to the layout definition data for which the second thumbnail image data has not been generated;
11. The display unit according to claim 1, wherein a second thumbnail image indicated by the second thumbnail image data is displayed on the display unit as the layout identification image corresponding to the layout definition data for which the second thumbnail image data has already been generated. 12. The terminal device according to any one of the above. A display unit, a storage unit, an operation unit that receives a user operation of touching a display screen of the display unit, a first communication unit that wirelessly communicates with the image recording apparatus and the service providing apparatus, and a first control unit. A recording system comprising: a terminal device; a recording unit capable of executing a recording process of recording an image on a sheet; a second communication unit; and an image recording device including a second control unit.
The storage unit stores first format information indicating a data format of first content data, which is content data obtainable from the service providing apparatus, in association with a first setting value used for the recording process. And the second format information indicating the data format of the second content data, which is the content data, and the second setting value used for the recording process are stored in association with each other, and the first format information and the second Two-format information is a different data format,
The first control unit includes:
A content image indicating the content data, a first content image indicating the first content data, and a layout information indicating an arrangement of the second content image indicating the second content data. Receiving layout definition data including the layout definition data from the service providing device through the first communication unit;
A content image display process for displaying the first content image and the second content image on the display unit in a state where a user operation for selecting the first content image and the second content image can be received by the operation unit; ,
In response to a user operation for selecting the first content image being received by the operation unit, the first content data indicated by the first content image indicated by the layout definition data acquired in the layout data acquisition process is displayed. A first content data acquisition process received from the service providing device through the first communication unit;
Displaying a first preview image on the display unit, the first preview image representing a result of executing the recording processing on the acquired first content data in accordance with the first setting value corresponding to the data format of the first content data; Preview display processing,
In response to the operation unit receiving a user operation for instructing execution of the recording process in which the result is represented by the first preview image, the first recording instruction information for instructing execution of the recording process is stored in the first storage unit . A first output process for transmitting to the image recording apparatus through one communication unit;
In response to a user operation for selecting the second content image being received by the operation unit, the second content data indicated by the layout definition data acquired in the layout data acquisition process is displayed. A second content data acquisition process received from the service providing device through the first communication unit;
Displaying a second preview image on the display unit, the second preview image representing a result of executing the recording processing on the acquired second content data in accordance with the second setting value corresponding to the data format of the second content data. Preview display processing,
In response to the operation unit receiving a user operation instructing execution of the recording process in which the result is represented in the second preview image, the second recording instruction information instructing execution of the recording process is transmitted to the second storage unit . A second output process for transmitting to the image recording apparatus through the 1 communication unit;
The second control unit, in response to receiving the first recording instruction information or the second recording instruction information from the terminal device through the second communication unit, the first recording instruction information or the second recording instruction A recording system that causes the recording unit to execute the recording process indicated by information. A display unit, a storage unit, an operation unit that accepts a user operation of touching a display screen of the display unit, a communication unit, and a computer, the program being executed by the computer of a terminal device including:
The communication unit is capable of wireless communication with an image recording apparatus and a service providing apparatus capable of executing a recording process of recording an image on a sheet,
The storage unit stores first format information indicating a data format of first content data, which is content data obtainable from the service providing apparatus, in association with a first setting value used for the recording process. And the second format information indicating the data format of the second content data, which is the content data, and the second setting value used for the recording process are stored in association with each other, and the first format information and the second Two-format information is a different data format,
The program is
A content image indicating the content data, a first content image indicating the first content data, and a layout information indicating an arrangement of the second content image indicating the second content data. Including layout definition data, a layout data acquisition process of receiving from the service providing device through the communication unit,
A content image display process for displaying the first content image and the second content image on the display unit in a state where a user operation for selecting the first content image and the second content image can be received by the operation unit; ,
In response to a user operation for selecting the first content image being received by the operation unit, the first content data indicated by the first content image indicated by the layout definition data acquired in the layout data acquisition process is displayed. A first content data acquisition process received from the service providing device through the communication unit;
Displaying a first preview image on the display unit, the first preview image representing a result of executing the recording processing on the acquired first content data in accordance with the first setting value corresponding to the data format of the first content data; Preview display processing,
In response to the operation unit receiving a user operation instructing execution of the recording process in which a result is displayed in the first preview image, the first recording instruction information instructing execution of the recording process is transmitted to the communication unit. A first output process for transmitting to the image recording device through the unit,
In response to a user operation for selecting the second content image being received by the operation unit, the second content data indicated by the layout definition data acquired in the layout data acquisition process is displayed. A second content data acquisition process received from the service providing device through the communication unit;
Displaying a second preview image on the display unit, the second preview image representing a result of executing the recording processing on the acquired second content data in accordance with the second setting value corresponding to the data format of the second content data. Preview display processing,
In response to the operation unit receiving a user operation instructing execution of the recording process in which a result is represented by the second preview image, the second recording instruction information instructing execution of the recording process is transmitted to the communication unit. And a second output process to be transmitted to the image recording apparatus through a unit.

Images