JP2021086414A - Information processing apparatus, information processing method, and program - Google Patents

Abstract

To provide an information processing apparatus which can prevent an unnecessary processing load from being generated when hybrid application is used for printing, an information processing method, and a program.SOLUTION: A print application 200 which executes an application with a script layer 217 and a native layer 218 in cooperation with each other includes: a content drawing unit 205 which generates a content to be printed including a plurality of elements; a rendering request unit 222 which issues a request for rendering a content to a web engine 224 which renders elements included in the content asynchronously, so as to render at least two elements after loading at least two of the elements included in the content; and a printer data generation unit 212 which acquires an image rendered by the web engine 224 to generate data to be printed.SELECTED DRAWING: Figure 2

Description

The present invention relates to an information processing apparatus, an information processing method, and a program that perform processing related to printing.

In recent years, portable multifunctional mobile terminals (mobile computers) equipped with a camera function have become widespread. Such a mobile computer is basically composed of three elements. That is, it is composed of hardware that is the computer itself, an operating system (OS) that runs on the hardware, and an application that runs on the OS.

As a form of an application that operates on such a mobile computer, there is a form called a so-called hybrid application. A hybrid application is an application that retains both the advantages of a native application and the advantages of a web application. Hereinafter, the application may be referred to as an "application".

Patent Document 1 describes a technique for generating data used for printing by a printing application in the form of a hybrid application. Patent Document 1 describes a technique in which an SVG content generated by a print application is drawn by a web engine provided as standard in the OS and imaged using the screenshot function of the OS.

JP-A-2018-5575

In the web engine provided as standard in the OS, when SVG content is read, each element of the SVG content is analyzed and output asynchronously from the state in which it can be reflected as image data. This is because when displaying web contents, it is said that usability is better if the screen is continuously updated even little by little because the waiting time for experiencing is shortened.

On the other hand, in the image for printing, the image in the process of being generated is often unnecessary. However, when a plurality of elements are included in the SVG content, the web engine draws the individual elements asynchronously, so that an area requiring redrawing may occur. Therefore, there is a risk of causing an unnecessary processing load on the computer.

The program according to one aspect of the present invention includes a first layer composed of a script instruction set that is translated and executed by a processor when an application is executed, and the processor as an instruction set that can be executed by the processor. A program having a second layer pre-compiled by different units and executing the application in cooperation with the first layer and the second layer, the processor, and a plurality of elements. With respect to the first generation means for generating the content to be printed to be included and the drawing means for drawing the elements included in the content asynchronously, at least two or more elements among the plurality of elements included in the content to be printed are present. A request means for requesting drawing of the content to be printed generated by the first generation means and a request by the request means so that the drawing means draws at least two or more elements after being loaded. Based on this, it functions as a second generation means that acquires an image drawn by the drawing means and generates data for printing.

According to the present invention, it is possible to prevent an unnecessary processing load from being generated when printing is performed using a hybrid application.

The block diagram which shows the example of the system including the information processing apparatus. Block diagram of the software configuration. A flowchart showing the user's operation procedure and internal processing inside the application. The figure which shows the SVG content. A flowchart showing the details of the printer information acquisition process. The figure which shows the example of the setting screen which selects a paper size. The figure which shows the example of the image to be drawn.

Hereinafter, embodiments will be described with reference to the drawings. It should be noted that the following embodiments do not limit the present invention, and not all combinations of features described in the present embodiments are essential.

<Explanation of hybrid app>
Prior to the explanation of this embodiment, a supplementary explanation of the hybrid application will be given. A hybrid app is an app that retains both the benefits of a native app and the benefits of a web app. The native application and the web application will be described below.

A native application is an application that is usually developed using a development environment and a development language prepared for each OS. For example, the C / C ++ language is used on the OS provided by the company A, the Java (registered trademark) language is used on the OS provided by the company B, and a different development language is used on the OS provided by the company C. Normally, a native application is precompiled in each development environment and converted from a so-called high-level language that can be understood by humans to an instruction set group such as an assembler that can be interpreted by a computer CPU. As described above, in a normal native application, since the CPU directly interprets the instruction, there is an advantage that high-speed operation is possible.

On the other hand, a web application is an application that runs on a web browser that is built into the OS on each computer as standard in recent years. Web applications are generally developed using HTML5, CSS, JavaScript, etc. so that they can be interpreted by a web browser. Since these are web standard languages, there is an advantage that once a web application is written in these web standard languages, it can be operated in an environment in which a web browser operates.

In a hybrid app, all or most of its user interface (UI) is executed by a script layer written in a standard web language such as HTML5, CSS3, or Javascript. On the other hand, processing for linking with the OS, such as calling an OS function, is executed by the native layer in the hybrid application. The native layer is developed in the native language according to the OS, and is pre-compiled by a unit different from the processor that executes the hybrid application. With this configuration, for example, it is possible to use a function written in a native language from a content written in a web standard language. In this way, the hybrid application has a configuration in which both the script layer and the native layer in the web standard language are included in one application. With such a configuration, it is possible to have a software configuration that retains both the advantages of the native application and the advantages of the web application. In addition, the hybrid application has an advantage that the development cost is low because the script layer can be commonly used for a plurality of platforms (OS and the like).

Here, it is assumed that a printing application in the form of a hybrid application is used to print content with a logo. The hybrid application first uses a web language to generate print content with a logo superimposed on the content. Then, at the time of printing, the print content described in the web language is converted into an image format (JPEG, BMP, PNG, PDF, etc.) that can be transmitted to the printer. For example, as the print content of the hybrid application using SVG, the following SVG can be mentioned as an example. SVG (Scalable Vector Graphics) is one of the description methods for displaying graphics, which can be used in a web standard language.
----------------------------------------------
[[SVG description example 1]]
<svg>
<image xlink: href = "photo"/>
<g> logo </ g>
</ svg>
----------------------------------------------

In SVG description example 1, detailed information is omitted for simplification. The SVG description example 1 represents an SVG content 400 including a photo 401 arranged on the entire surface and a logo 402 superimposed on a part of the photo 401, as shown in FIG. 4A described later. As described above, in SVG content, there may be content composed of a plurality of elements.

When the SVG content as in SVG description example 1 is generated by the print application in the form of a hybrid application, the web engine provided as standard in the OS draws the SVG content. Then, the print application transmits the print data to the printer using the image data obtained by imaging using the screenshot function of the OS, and the printer prints.

Here, in the web engine provided as standard in the OS, when SVG content such as SVG description example 1 is read, each element of the SVG content is analyzed and can be reflected as image data. Is output asynchronously from. This is because when displaying web content, it is said that usability is better if the screen is continuously updated even little by little because the waiting time for experiencing is shortened.

For example, as in SVG description example 1, it is assumed that there is SVG content composed of a plurality of elements, and an image in which the plurality of elements are superimposed is drawn. Then, the area of the element located at the backmost surface is covered with the element located at the frontmost surface, so that it is originally an area that does not require drawing. In the example of FIG. 4A, the area of the photograph 401 located on the back surface of the logo 402 is originally an area that does not require drawing. If drawing is performed from the element existing at the back of the SVG content, the overlapping area will be redrawn every time a new element is drawn. Further, when they have a transparency setting, it is necessary to calculate a pixel value in consideration of the transparency attribute each time, which increases the processing load.

Therefore, in the embodiment described below, when a print application in the form of a hybrid application is used, an example of controlling the processing so as not to cause an unnecessary processing load on the computer will be described.

<Explanation of hardware configuration>
FIG. 1 is a block diagram showing an example of a system including the information processing apparatus 115 of the present embodiment. The information processing device 115 is, for example, a portable information terminal such as a smartphone or a mobile phone. The information processing device 115 includes a CPU 100, a ROM 101, a RAM 102, a secondary storage device 103, a display 104, a touch sensor 105, and an acceleration sensor 106. It also has an external I / F 107-109, an internal imaging device 110, a control bus / data bus 116, and a battery 117. The information processing device 115 may not have all the configurations shown in FIG. 1, or may have configurations other than those shown in FIG.

The CPU 100 is a central processing unit (processor), and executes various processes described below according to a program. Although the number of CPUs 100 is one in FIG. 1, it may be composed of a plurality of CPUs or CPU cores. The ROM 101 stores a program executed by the CPU 100. The RAM 102 is a memory for temporarily storing various information when the CPU 100 executes a program.

The secondary storage device 103 is a storage device such as a hard disk or a flash memory, and stores data such as a database that holds processing results such as a file or image analysis, and various programs. The display 104 displays various information such as a UI (user interface) for receiving operations for realizing various processes and processing results of the executed processes. The display 104 may include a touch sensor 105.

The image data obtained by imaging by the internal imaging device 110 is stored in the secondary storage device 103 after undergoing predetermined image processing. The image data can also be read from the external imaging device 111 connected via the external I / F (interface) 108.

The information processing device 115 includes an external I / F 109 and can perform communication via a network 113 such as the Internet. The information processing device 115 can also acquire image data from the server 114 connected to the network 113 via the external I / F 109.

The information processing device 115 includes an acceleration sensor 106, and can acquire acceleration information regarding the position and orientation of the information processing device 115 itself. The information processing device 115 is connected to the printer 112 via an external I / F 107, and can output data such as image data to the printer 112.

The printer 112 is also connected to the network 113, and can send and receive data necessary for the printer via the external I / F 109.

The external I / Fs 107 to 109 are interfaces having at least one of wired communication and wireless communication, and communicate with an external device (printer 112 or server 114) according to the communication mode to be used. Wired communication includes, for example, USB, Ethernet (registered trademark), and wireless communication includes wireless LAN, NFC, Bluetooth (registered trademark), infrared communication, and the like. Further, when using a wireless LAN as wireless communication, there are a form in which the devices are directly connected to each other, and a form in which the devices are connected via a relay device such as a wireless LAN router. Further, although the external I / Fs 107 to 109 are separately configured in the figure, they may be integrally configured.

The power supply required for the operation of the information processing apparatus 115 is supplied by the battery 117. The various components included in the information processing device 115 are connected to each other via the control bus / data bus 116, and the CPU 100 controls the various components via the control bus / data bus 116.

In the present embodiment, the information processing device 115 serves as an execution place (software execution environment) for software such as a program executed by the control unit (CPU 100) included in the information processing device 115.

<Software block diagram>
FIG. 2 is a block diagram of a software configuration that operates in the information processing device 115. The information processing device 115 executes the programs of the script layer 217, the native layer 218, and the OS layer 219. Each of these layers is realized by the CPU 100 reading and executing the corresponding program stored in the ROM 101 or the secondary storage device 103.

The script layer 217 includes an image acquisition unit 201, a content drawing unit 205, an image processing control unit 206, a data conversion unit 207, a content operation unit 210, a printer control unit 211, and a rendering request unit 222.

The native layer 218 includes an image reading unit 202, a data conversion unit 203, a data holding unit 204, an image processing unit 208, a printer data generation unit 212, a rendering unit 216, a data storage unit 220, and an external device communication unit 221.

The OS layer 219 includes a touch event 209, a printer communication unit 213, an interpreter 214, and a state detection unit 223. Interpreter 214 has a web engine 224.

The script layer 217 (first layer) describes an instruction set (drawing content, displaying an image, playing a moving image, etc.) with text data using a web standard language such as HTML5, CSS3, or Javascript. It is a program layer. This instruction set is called a script instruction set, and the script instruction set is an instruction set that can be executed by a processor (for example, CPU 100). In the script layer 217, various instruction sets of text data are translated and executed on the application execution environment by using the processor (CPU100) existing in the application execution environment. The execution form includes dynamically translating the instruction statement line by line each time it is executed, translating it when the application is started, or translating it when the application is installed on the information processing device 115. Conceivable.

Hereinafter, the processing by the script layer 217 and the processing content of the script layer 217 will be referred to as a "script". An example of a form in which a script instruction is translated in the information processing apparatus 115 is to use an interpreter function provided in the native layer 218 or the OS layer 219. In this embodiment, it is assumed that most of the UI of the application is described in the script layer 217.

The native layer 218 is a program layer in which an instruction set pre-translated (compiled) other than the application execution environment is described. As a form, code written in a high-level language such as C or C ++ is compiled in advance on the PC or server of the developer of the application, and is a set of instructions that can be interpreted by the CPU 100. Hereinafter, processing by the native layer 218, processing contents of the native layer 218, and calling the functions of the OS layer 219 described later from the native layer 218 are referred to as "native". As another mounting system of the native layer 218, Java (registered trademark) can be mentioned. Java is a high-level language similar to C / C ++, and is translated into intermediate code in advance in the development environment at the time of application development. The translated intermediate code operates in the Java virtual environment provided in each OS. In the present embodiment, such a program form is also included in a kind of native layer 218.

The OS layer 219 corresponds to the operating system (OS) of the information processing device 115. The OS layer 219 has a role and a unique function of providing the application with the use of the hardware function. The OS layer 219 includes an API, and the script layer 217 and the native layer 218 can use the functions of the OS layer 219 via the API.

In the present embodiment, enabling the call of the native layer 218 from the script layer 217 is referred to as "binding" or "binding". The native layer 218 includes an API, and the script can use various native functions by calling the API. Such a binding function is usually a function that various OSs have as standard.

In the present embodiment, an application including a script layer 217 and a native layer 218 is referred to as a hybrid application. The hybrid application is also an application that uses the functions of the OS layer 219. The hybrid application in the present embodiment is a printing application 200 capable of superimposing a second image such as a logo on a first image such as a photograph and causing the printer 112 to print. In addition, the print application can also preview the superimposed image. The above is an overview of each layer of software. Hereinafter, each block of FIG. 2 will be described.

The image acquisition unit 201 of the script layer 217 requests the native layer 218 to acquire image data to be printed, for example. The request of the image acquisition unit 201 includes, for example, a method of designating an image path, a method of prompting a dialog display, and the like.

The image reading unit 202 of the native layer 218 acquires image data from the image data group 215. The method of acquiring the image data from the image data group 215 depends on the request method of the image acquisition unit 201 of the script layer 217. The request method includes selecting from the dialog box provided on the UI, or selecting the image directly from the image path. When reading an image, the image reading unit 202 may return the image path to the script layer 217, or may return the image path to the script layer 217 in Base64 format. In addition, when a unique ID is assigned to the image and the image is specified, a form using this ID may be used. Further, the image reading unit 202 may display a list of images stored in the secondary storage device 103 on the OS layer 219. Then, the image selected by the user in the list display may be read via the OS layer 219.

The data conversion unit 203 of the native layer 218 converts the data of the native layer 218 into data in a format that can be used by the script layer 217. The data in the native layer 218 is, for example, image data or parameters in binary format. The format data that can be used in the script layer is, for example, text format (Base64) image data or JSON. JSON is an abbreviation for Javascript Object Notation. On the other hand, the data conversion unit 203 of the native layer 218 also converts the data sent from the script layer 217 into data in a format that can be used by the native layer 218.

The data conversion unit 207 of the script layer 217 uses the data of the script layer 217 (example: text format processing parameters) as the data in a format that can be used by the native layer 218 (example: text format (JSON format) processing parameters or character strings). Convert to.

The data holding unit 204 of the native layer 218 holds the image data read by the image reading unit 202, the image data processed by the image processing unit 208, and the divided image group. The retained image data is expanded into, for example, RGB image data, and is in a format in which image processing can be executed.

The content drawing unit 205 of the script layer 217 describes the content for printing by using a web standard language. This description also reflects the script edited and operated by the content operation unit 210. The content script described by the content drawing unit 205 is interpreted by the interpreter 214 of the OS layer 219 and displayed on the display 104. Recent PCs and mobile computers basically include an interpreter 214 that analyzes HTML and Javascript. The interpreter 214 has a web engine 224, which is a web language interpreter. In this example, unless otherwise stated, the web engine is used interchangeably with the rendering engine or web browser.

The image processing control unit 206 of the script layer 217 determines the correction parameters used for image processing in the native layer 218 and the image to be processed, and requests the image processing unit 208 of the native layer 218 to perform image processing. The correction parameter is converted into a format that can be transmitted to the native layer by the data conversion unit 207 of the script layer 217, and then transmitted to the native layer 218.

The image processing unit 208 of the native layer performs image processing on the image specified by the image processing control unit 206. At that time, what kind of image processing is performed is determined by the parameters set by the image processing control unit 206. Regarding the designation of the image, in addition to the method of using the ID as described above, a method of receiving the image path from the script layer, a method of receiving the entire image data, and the like can be considered.

In the method using the image ID, the image and the image ID are transmitted from the native layer 218 to the script layer 217. Then, the image is not transmitted from the script layer 217 to the native layer 218, and the image ID and the parameters set by the image processing control unit 206 may be transmitted. Then, the image reading unit 202 of the native layer 218 reads the image specified by the image ID, and the image processing control unit 206 executes image processing on the read image according to the parameter. If the image read by the image reading unit 202 is a file containing an image having a large number of pixels and a thumbnail image having the same content and a small number of pixels, even if the thumbnail image is transmitted from the native layer 218 to the script layer 217. Good. Then, as an image for which the image processing control unit 206 executes image processing, the image reading unit 202 may read the image according to the image ID. As a result, it is possible to prevent an image having a large number of pixels and a large amount of data from being transmitted from the native layer 218 to the script layer 217.

The touch event 209 of the OS layer 219 acquires information regarding the touch of the display 104. Examples of the touch-related information include touch detection of the display 104, touched position information, and the like. The acquired information regarding the touch is transmitted to the content operation unit 210 of the script layer 217 via the native layer 218.

The content operation unit 210 of the script layer 217 changes the script command in order to reflect the image data when the image data is returned from the native layer 218 as a result of the image acquisition request, for example. For example, as will be described later, it is converted into a hierarchical model called a DOM (Document Object Model) tree. Further, the content operation unit 210 changes the script instruction so that the content is not drawn asynchronously by the web engine 224. Details will be described later.

The printer control unit 211 of the script layer 217 controls the request for printer detection, the display of the printer setting screen, and the generation and transmission of print information. On the printer setting screen, the printer settings such as paper size, paper type, and color / monochrome are set. Based on the items set here, the printer data generation unit 212 of the native layer 218 generates printer data.

The printer data generation unit 212 of the native layer 218 generates data and commands necessary for printer communication based on a request from the printer control unit 211 of the script layer 217. The data required for printer communication is data according to the communication protocol. The command required for printer communication is data for determining the operation of the printer, such as printing or scanning. In this way, the printer data generation unit 212 generates printer data including commands for determining the operation of the printer. The command for determining the operation of the printer includes image data for printing. It also includes a command that asks the printer to send printer information.

The external device communication unit 221 of the native layer 218 is an interface (IF) for communicating with an external device connected to the information processing device 115 such as a printer. The external device communication unit 221 of the native layer 218 transmits the data received from the printer data generation unit 212 and receives the information from the printer 112. In the present embodiment, the external device communication unit 221 communicates via the printer communication unit 213 of the OS layer 219, but the external device communication unit 221 may directly transmit data to the external IF 107. If the printer communication unit 213 of the OS layer 219 supports the communication protocol used by the external device, the external device communication unit 221 may use the function. If the printer communication unit 213 does not support the communication protocol used by the external device, the external device communication unit 221 communicates according to the communication protocol.

The interpreter 214 of the OS layer 219 interprets and executes the instruction written in the web standard language generated by the script layer 217. For example, instructions such as drawing an image are executed through interpreter 214 (web engine 224).

The image data group 215 is an area that holds image data. The data storage unit 220 functions to store the image data held by the data storage unit 204 in the image data group 215, if necessary.

The rendering request unit 222 of the script layer 217 requests the rendering unit 216 of the native layer 218 to render (draw). At that time, the rendering requesting unit 222 converts the content described by the content drawing unit 205 by the data conversion unit 207 and then delivers it to the rendering unit 216. If there is a necessary process before passing it to the rendering unit 216, the rendering requesting unit 222 may edit the content description by using the content operation unit 210. If editing the content description directly affects the UI, a duplicate of the content description may be generated and the duplicated content description may be edited.

The rendering unit 216 performs rendering for generating image data for transmission to the printer from the contents received from the rendering requesting unit 222. This rendering includes, for example, generating an image at the output resolution to the printer 112 at the script layer 217. In the process in which the rendering unit 216 generates an image from the content, the rendering unit 216 also utilizes the function of the web engine 224. The rendering result in the native layer 218 and the image being generated are not displayed on the display 104.

The state detection unit 223 of the OS layer 219 detects the state of the device being used. For example, when there is another notification such as an incoming call while using the application, the occurrence or state change of the event is transmitted to the native layer 218.

<Flowchart>
Hereinafter, a procedure in which a smartphone, which is a type of mobile computer, generates SVG content and a procedure for converting the generated SVG content into an image and instructing the printer 112 to print it will be described.

FIG. 3 is a flowchart of the present embodiment. FIG. 3A is a flowchart showing a user operation procedure for the application, and FIG. 3B is a flowchart describing internal processing inside the application corresponding to the user operation. There is also a case where the user operation and the internal processing have a one-to-many relationship. For example, in the paper size selection of S304, two processes, S304_1 and S304_2, are performed as internal processes. In this way, when a plurality of internal processes occur for one user operation, a sub number is added with an underscore in addition to the operation procedure number. The process of FIG. 3B is realized when the CPU 100 reads the program stored in the ROM 101 or the secondary storage device 103, expands the program into the RAM 102, and the CPU 100 executes the program. For example, the CPU 100 functions as a print application 200. The symbol "S" in the description of each process means a step in the flowchart.

In the present embodiment, it is assumed that a content (also referred to as SVG content) in which one logo is superimposed on one photo as shown in FIG. 4A is finally generated and printed. The SVG content 400 of FIG. 4A shows an example in which the logo 402 is superimposed on a part of the front surface of the photo 401. FIG. 4B shows an example of a UI screen displayed on the display 104 of the information processing apparatus 115 by the print application 200. In the following, first, the generation of SVG content will be described.

As described above, the print application 200, which is a hybrid application, generates the content on which the logo is superimposed by using the web language. In this embodiment, an example of generating print content with SVG that can be used as a part of HTML will be described. Of course, print content may be generated by HTML.

Recent printers can print with very high image quality, and accordingly, the image size required for printing requires a very large size such as 2000 px × 3000 px. This size is so large that it extends beyond the screen of the smartphone. Therefore, in the process of generating SVG content, the content is basically displayed on the screen of the display 104 in a small size for UI. In this example, the SVG size displayed for the UI will be 600px × 400px. The template of 600px x 400px SVG content is as follows.
----------------------------------------------
[[SVG description example 2]]
<svg width ＝ 400 height ＝ 600 viewBox ＝ ”0 0 400 600”>
</ svg>
----------------------------------------------

The flow chart of FIG. 3 starts processing when the print application 200 reads the template of the SVG content. Specifically, in S301, the user activates the print application 200, and the print application 200 reads the template of the SVG content (S301_1). In S301_1, for example, a screen in which the SVG content 400 in FIG. 4B is not displayed on the screen of the display 104 is displayed.

SVG is a kind of XML (Extension Markup Language), and a group from a specific start tag (<svg>) to an end tag (</ svg>) is an element. Content is generated by adding elements.

In S302, the user adds an image as part of the content. Here, it is assumed that Photo 401 is added. That is, the print application 200 adds an image to the SVG content based on the user operation in S302_1. The image is added by the user selecting an image from the smartphone terminal, triggered by pressing the image addition button 403 (FIG. 4B). For image selection and acquisition, the API of the native layer prepared by each OS may be used by using the binding function, or the HTML standard function may be used. For example, in the case of HTML, it is well known that file access and data acquisition can be performed in the terminal by using an input tag such as <input type = "file" or less />. Of course, the image may be acquired from the SNS, the cloud, or the website through the network.

When the acquired image is added to the SVG content, the above SVG description example 2 becomes as follows.
----------------------------------------------
[[SVG description example 3]]
<svg width ＝ 400 height ＝ 600 viewBox ＝ ”0 0 400 600”>
<image width = 400 height = 600 xlink: href = "image data"></image>
</svg>
----------------------------------------------

The image element is inserted between the start tag and the end tag of SVG. XML can describe contents in such a hierarchical structure. In the case of this example, the image element can be expressed as a child element of the svg element. Conversely, the svg element can be expressed as the parent element of the image element. In this way, the elements described in the hierarchical structure are expressed as child elements, parent elements, and the like. Various settings such as color or font type can be set for the SVG element. At this time, the parent element is not affected by the set value of the child element, and the child element is configured to be affected by the set value of the parent element.

In the SVG description example 3, the image element is set with a width (wise) of 600 px and a height (height) of 400 px. Since the same numerical value is set for SVG, the photograph is displayed on the entire surface of the SVG content in this SVG. The "image data" in the image element may be a path indicating an image or an image in Base64 format.

Here, the DOM (Document Object Model), which is a concept in a web browser, will be described. The DOM is a model for managing contents such as SVG described in XML. The SVG content as described above is just a character string and cannot be operated by Javascript as it is. Therefore, the web browser analyzes XML such as HTML or SVG, converts it into a model having a hierarchical structure called a DOM tree, and manages it. As a result, the elements of SVG can be accessed from Javascript, and operations such as changing the contents of SVG can be performed.

In this embodiment, the modified SVG is represented by a character string in XML format for the sake of simplicity. In the actual application, Javascript is executed and the element is added as DOM. That is, when an image is added to the empty SVG content shown in SVG description example 2, Javascript is executed and an element is added as a DOM. Specifically, the content operation unit 210 executes "generation of image element", "add setting value to image element", and "add to svg" with Javascript. For example, adding an element to SVG content is implemented as follows.
----------------------------------------------
[[Example of Javascript execution]]
var xmlns = “http: //www.w3.org/2000/svg” ；
var xlink = “http: //www.w3.org/1999/xlink”
var img = document. createElementNS (xmlns, “image”) ；
img. setAttributeNS (xlink, “href”, “image data”) ；
img. setAttribute (“width”, 400) ；
img. setAttribute (“height”, 600) ；
var svg = document. querySelector (“svg”) ；
svg. appendChild (img) ；
----------------------------------------------

Next, in S303, the user presses the logo addition button 404 in FIG. 4B to select a logo, and adds the logo 402 to the photo 401 added in S302 as shown in FIG. 4A. That is, in S302_1, the print application 200 adds the logo 402. The logo can be added by the same procedure as in S302_1. However, the logo shall be displayed in a small size at the lower right of the screen, and it may be represented as SVG as shown below.
----------------------------------------------
[[SVG description example 4]]
<svg width ＝ 400 height ＝ 600 viewBox ＝ ”0 0 400 600”>
<image width = 400 height = 600 xlink: href = "image data"></image>
<image x = 300 y = 550 width = 100 height = 50 xlink: href = "image data"></image>
</svg>
----------------------------------------------

In SVG description example 4, "x" and "y" are newly introduced. These are the x-coordinate and the y-coordinate, and the two-dimensional plane is translated by the specified numerical value. As a result, the SVG content 400 in which the logo 402 is superimposed on the lower right position on the front surface of the photo 401 is completed.

Next, the print application 200 communicates with the printer 112, determines the paper size used for printing, and performs a process of acquiring the number of pixels of the image required for rendering. First, the paper size is selected. In S304, the user presses the print setting button 405 shown in FIG. 4 (b). In response to this, the print application 200 acquires printer information from the printer in S304_1, and the print application 200 displays the size list on the display 104 in S305_2. The details of the processing of S304_1 and S305_2 will be described with reference to FIG.

<Details of print settings>
FIG. 5 is a flowchart showing the details of the printer information acquisition process. S501 and S509 to S511 are processes executed by the CPU 100 of the information processing apparatus 115 in the script layer 217. Further, S502, S504, S505, S507, and S508 are processes executed by the CPU 100 of the information processing apparatus 115 in the native layer 218. Further, S503 and S506 are processes executed by the printer 112, which is an external device.

In S501, the script layer 217 requests the native layer 218 to acquire printer information. The request method is the same as when selecting an image, the native API is called from the script by the binding function. The native layer 218 is provided with a so-called wrapper that can be called directly from the script layer 217 or indirectly calls the function. For example, a native function called GetPrinterInfo is prepared, and the function is called from the script side. In this way, the native layer 218 acquires the request for communication with the external device from the script layer 217.

Normally, the script layer 217 cannot directly communicate with an external device due to security restrictions. Therefore, as described above, the script layer 217 once requests the native layer 218 to acquire the external device information, and communicates with the external device via the native layer 218. The native layer 218 has a function of communicating with an external device (for example, a printer 112) via the OS layer 219.

When the above function is called, the native layer 218 performs printer detection, so-called discovery, in S502. For example, the native layer 218 detects printers connected by the same wireless LAN router. Here, in order to detect a printer that can communicate, for example, a protocol such as Bonjour (registered trademark) is used to attempt communication with the printer using a method such as broadcast or multicast.

In S503, the printer 112 responds to the request sent in S502.

In S504, the native layer 218 detects and stores the IP address of the printer that responded. Next, in S505, the native layer 218 requests the IP address of the printer that responded to provide the printer information. If there are multiple printers responding, the native layer 218 requests all printers to provide information. In the native layer 218, a command for acquiring printer information is generated. The command is a command that specifies the operation of the printer, and as an example, is a command expressed in XML as shown below.
----------------------------------------------
[[Printer information acquisition command example]]
01: <? Xml version = "1.0" encoding = "utf-8"?>
02: <cmd xmlns: trans = "http: //www.xxxx/yyyyy/">
03: <contents>
04: <operation> GetInformation </ operation>
05: </ contents>
06: </ cmd>
----------------------------------------------

It should be noted that the numerical values such as "01:" written on the left side of each of the above lines are line numbers added for explanation and are not described in the original XML format text.

The first line is a header, which indicates that it is described in XML format. The cmd on the second line means the start of the command. The namespace is specified by xmlns, and the definition of command interpretation is specified. The end of the command is indicated by </ cmd> on the 6th line.

The third line is a declaration that describes the contents thereafter, and the fifth line indicates the end.

The required instruction is described on the fourth line, and the actual instruction wording exists between <operation> and </ operation>. The command wording, Get Information, is a command for acquiring information on a printer, which is an external device. For example, it is required to provide capability information such as the paper type, size, borderless printing function, and print quality supported by the printer.

To generate the printer information acquisition command, for example, a fixed text stored in advance in the ROM 101 may be read. Further, the description is not limited to a text format such as XML, but may be described in a binary format and communication may be performed using a protocol according to the description. The generated printer information acquisition command is transmitted to the printer 112 via the external device communication unit 221 in a format according to a communication protocol supported by the printer such as HTTP.

Moreover, the communication method is not limited to this. Wi-Fi Direct (registered trademark), Bluetooth (registered trademark), infrared communication, telephone line, wired LAN, or connection using USB may be used, and the same effect can be obtained by communicating using a protocol according to the method. Can be done.

In the example of FIG. 5, the native layer 218 is configured to generate commands, but the script layer 217 is configured to generate commands, and the same effect can be obtained. In that case, the script layer 217 generates, for example, a command including the above XML format statement and passes it to the native layer 218. In response, the native layer 218 may send the command to the printer 112 in a format according to the communication protocol.

When the printer 112 receives a command from the information processing device 115, the printer 112 transmits the printer information, which is the device information, to the information processing device 115 in the XML format according to the communication protocol in S506. An example of printer information is shown below.
----------------------------------------------
[[Example of printer information]]
01: <? Xml version = "1.0" encoding = "utf-8"?>
02: <cmd xmlns: trans = "http: //www.xxxx/yyyyy/">
03: <contents>
04: <device id = ”Printer001” />
05: <memory receive = 7680000 />
06: <mode = 1>
07: <media> Glossy Paper </ media>
08: <size> A4 </ size>
09: <quality> 1 </ quality>
10: <border> no </ border>
11: <pixels>
12: <width> 4000 </ width>
13: <height> 6000 </ height>
14: </ pixels>
15: </ mode>
16: <mode = 2>
~ Omitted ~
</ Mode>
<Mode = 3>
~ Omitted ~
</ Mode>

~ Omitted ~

</ Contents>
</ cmd>
----------------------------------------------

The first line is a header, which indicates that it is described in XML format. The cmd on the second line means the start of the command. The namespace is specified by xmlns, and the definition of command interpretation is specified. The bottom line </ cmd> indicates the end of the command.

The third line is a declaration that describes the content after that, and the content continues until </ contents> below. The fourth line shows the device ID. Here, it means that the model name of the printer 112 is "Printer001". The sixth and subsequent lines are descriptions of each mode of the printer 112. Information in one mode is described from <mode> to </ mode>. In the sixth line, the mode number is 1. Subsequent <media> describes the type of printing paper, <size> describes the paper size, <quality> describes the print quality, and <border> describes information with / without borders. <Pixels> on the 11th line represents the size of the input image required for printing, and indicates that an image of 4000 [px] in the horizontal direction and 6000 [px] in the vertical direction is required at the time of printing. ..

From the 15th line onward, information about the other mode, mode 2, is described. In this way, the model name of the printer 112 and all the modes supported by the printer are described in XML. The method of describing the printer information is not limited to this, and may be a text other than the tag format or another format such as a binary format.

Further, the example of the printer information is not limited to the one shown above. For example, status information such as image processing and analysis processing that can be processed by the printer, the presence / absence of a quiet printing mode, the presence / absence of a memory card, and the remaining amount of ink may be passed. Examples of image processing include color conversion such as monochrome, sepia, or saturation enhancement, multiple image layout, white balance correction, noise removal, or other processing that automatically corrects a photo to a preferred color or brightness. Be done.

In S507, the native layer 218 acquires printer information from the printer 112. That is, the native layer 218 acquires, for example, the type, size, print quality, bordered / non-rimmed items, number of items, and the like of the printing paper in all modes of the printer 112 from the received printer information.

In S508, the native layer 218 converts the received printer information into a format that can be interpreted by the script layer 217, and transmits the received printer information to the script layer 217. Specifically, a native function is provided and the binding function is used. Alternatively, a method such as transmitting with the received XML format printer information or changing to a untagged text format may be used. Information may be provided as a return value each time a specific native function is called from the script layer 217. You may pass an argument such as the mode to be acquired to the native function and provide the information as the return value. The above-mentioned JSON character string may be used for delivery, or the data conversion units 207 and 203 may be used for delivery using a character string such as Base64.

In this embodiment, it is transmitted to the script layer as JSON (Javascript Object Notation) format. For example, the information is as follows.
----------------------------------------------
[[Example of JSON format]]
PrinterInfo = {
“DeviceID”: “Printer001”,
“MemoryRecieve”: “7680000”,
“Mode”: [{
“Media”: “Glossy paper”,
“Size”: “A4”,
“Quality”: 1,
“Border”: “no”,
“Pixels”: {
“Width”: 4000,
“Height”: 6000
}
},
// Array as many as the number of modes
]
}
----------------------------------------------

In S509, the script layer 217 displays a setting screen for selecting a paper size based on the printer information received from the native layer 218. FIG. 6 is a diagram showing an example of a setting screen for selecting a paper size. In this embodiment, information is acquired from the printer found first and displayed. Of course, if there are a plurality of printers that can be connected, the printer name may be displayed and the user may be allowed to select the printer to print. The selection of the printer is not limited to the above, and a method such as selecting a printer having more functions or a printer in which the print job is not congested can be considered.

As a method of forming the paper size setting screen as shown in FIG. 6, the radio button function of HTML may be used. By using a radio button, it is possible to easily construct a UI in which only one check 601 can be checked from a plurality of selection candidates 602.

Words and expressions such as "L version" and "A4" in FIG. 6 are determined by the native layer 218 in association with the information sent from the printer. For example, if the value of the paper size extracted from the information sent from the printer is "L", the text to be displayed is determined to be "L version". As a method of determination, the native layer 218 may hold these correspondence tables in advance and determine the text according to the correspondence table. By switching this correspondence table, it is possible to support multiple languages.

In the above, only the paper size is set, but other settings such as paper type, print quality, bordered / non-rimmed, double-sided / single-sided, color / monochrome, or image correction on / off are made. You may. In addition to the printing function as described above, status information such as image processing or analysis processing that can be processed by the printer, the presence / absence of a quiet printing mode, the presence / absence of a memory card, or the remaining amount of ink is displayed. May be good.

In S510, the script layer 217 accepts a paper size selection from the user via the displayed radio button. That is, the user puts a check 601 in the radio button from the plurality of selection candidates 602 and presses the enter button 603.

In S511, the paper size is determined according to the selection by the user, and the process ends. When the decision button 603 is pressed, the print application 200 determines the input image size associated with the determined paper size as the rendering size. The above is the description of FIG. Next, the description will be continued by returning to FIG.

In S305, the user presses the print button 406 and inputs a print instruction. Upon receiving the print instruction, the print application 200 starts the print process. When the printing process is started, the processes (S305_1 to S305_7) from the imaging of the generated SVG content to the printing are executed in order.

In S305_1, the print application 200 modifies the SVG size. That is, the content operation unit 210 converts the size of the SVG generated for the UI into the size for printing. The size for printing is determined based on the information determined on the paper size setting screen. In the present embodiment, as described above, the A4 size is selected, and the final output size of the image generated by rendering is 4000 px in width and 6000 px in height. Rendering means generating image data to be used for printing from SVG contents. In the present embodiment, unless otherwise specified, rendering and drawing are used with the same meaning. As will be described below, in the present embodiment, printing is performed using the image data obtained by taking a screenshot of the drawn image.

In this embodiment, a marker for drawing determination is attached to the SVG content. Markers are added to the SVG content so that the image obtained by screenshot before the completion of drawing the SVG content is not used as an image for printing. That is, the completion of drawing is determined by the presence or absence of the marker. In the present specification, drawing (rendering) indicates a state in which the analyzed SVG content can be output as image data, and it does not matter whether it is displayed or hidden on the display.

The SVG content transmitted from the script layer 217 to the native layer 218 is an SVG of 4005px × 6000px including the marker. Here, the SVG to which the marker is added is as follows.
----------------------------------------------
[[SVG description example 5]]
<svg width ＝ 4005 height ＝ 6000 viewBox ＝ ”0 0 4005 6000”>
<svg width ＝ 4000 height ＝ 6000 viewBox ＝ ”0 0 400 600”>
<image width = 400 height = 600 xlink: href = "image data"></image>
<image x = 300 y = 550 width = 100 height = 50 xlink: href = "image data"></image>
</svg>
<rect x ＝ 4000 y ＝ 0 width ＝ 5 height ＝ 6000 fill ＝ ”red”></rect>
</svg>
----------------------------------------------

FIG. 7 is a diagram showing an example of an image to be drawn. FIG. 7A shows an example of an image that is finally desired to be acquired as a screenshot. In the above SVG, as shown in FIG. 7A, the width is set to be 5 px wider than the print size, and the color of the area is all red. This expanded area corresponds to a marker.

In S305_2, the print application 200 modifies the SVG content. That is, the content operation unit 210 modifies the SVG content so that the asynchronous processing of the SVG content is not performed. The asynchronous process is a process in which each element of the SVG content is analyzed at any time when drawing is performed by the web engine 224, and the element is asynchronously drawn and output from the element that can be reflected as image data. When asynchronous processing is performed, unnecessary images are generated at any time as print images, which causes a wasteful load. In this embodiment, the SVG content is modified in order to control asynchronous processing. For example, in the present embodiment, in order to cause the web engine 224 to perform synchronous drawing (simultaneous drawing) of data, a description for controlling display ON / OFF, which is a setting item for the web engine 22, is embedded in the SVG content. The web engine 224 has a function of displaying a web page, and when displaying the web page, the web engine 224 performs drawing processing to perform image formation as described above. In the present embodiment, by adding a command not to be displayed to the SVG content, the web engine 224 is controlled so that the drawing process is not immediately performed, that is, the asynchronous process is not performed. The SVG to which such a modification is applied to the above-mentioned SVG description example 5 is as follows.
----------------------------------------------
[[SVG description example 6]]
<svg width ＝ 4005 height ＝ 6000 viewBox ＝ ”0 0 4005 6000” style ＝ ”display: none ；” onload ＝ ”(function () {this.style.display ＝'block' ；} .bind (this)) ( ) ； ”>
<svg width ＝ 4000 height ＝ 6000 viewBox ＝ ”0 0 400 600”>
<image width = 400 height = 600 xlink: href = "image data"></image>
<image x = 300 y = 550 width = 100 height = 50 xlink: href = "image data"></image>
</svg>
<rect x ＝ 4000 y ＝ 0 width ＝ 5 height ＝ 6000 fill ＝ ”red”></rect>
</svg>
----------------------------------------------

In the above SVG, a setting that does not display on the screen is added by adding a command of "display: none". As will be described later, in the present embodiment, the image data obtained by the screenshot of the image displayed on the off-screen is treated as the data for printing. In the web engine 224 in which drawing for each element is performed asynchronously, an image is drawn for each element as soon as the analysis of the element is completed. On the other hand, in the web engine 224, when drawing in a state where the arrangement relationship of all the elements can be specified, the drawing process is performed in consideration of the arrangement relationship. Therefore, by controlling the web engine 224 so as not to perform asynchronous processing, unnecessary drawing processing is not performed, and the processing load of the information processing device 115 can be reduced.

If "display: none" is added as in the above SVG, the drawing itself is not performed in the web engine 224. That is, in the web engine 224, the element in which "display: one" is described does not exist when the image is acquired. This hidden instruction is, in more detail, interpreted by the interpreter 214 and implemented by the web engine 224.

Further, in the above SVG, a script for canceling the non-display setting is embedded at the timing when "onload" (reading is completed) is detected. That is, an instruction to change to "display: block" is embedded. Note that display: block means to display an element. SVG has a function to cooperate with Javascript. The onload is issued from the web browser (web engine 224) when the DOM analysis and resource acquisition of the element are completed and the Javascript operation and drawing for the element are ready. That is, onload is a load completion notification issued from a web browser (web engine 224). By associating a script with onload as in the above example, it is possible to execute an arbitrary script at the timing of issuing onload.

Note that onload is issued when the loading of child elements (both image elements) is completed, but in the case of parent element onload, it is issued after all child element onloads are completed. In this example, the processing in the case of onload of the parent element is specified. Therefore, the onload of the parent element, which is issued after all the onloads of the child elements are completed, means that the drawing for printing is ready. That is, in the present embodiment, onload issuance is treated as a trigger in which preparation for drawing synchronization processing is completed.

Normally, the web engine 224 is configured to asynchronously draw the elements of the content from the viewpoint of usability. If the web engine 224 draws asynchronously from the element that has completed the analysis, unnecessary drawing processing will be executed many times, which affects the performance. For example, although it is desired to acquire the image of FIG. 7 (a), the logo is not reflected as shown in FIG. 7 (b), so to speak, after passing through the generation of an image in the process of being generated, in FIG. 7 (a). An image will be generated. FIG. 7B shows an example of an image in the process of being generated in which only the image of the photograph is reflected. Here, the logo area of FIG. 7B is not a desired image area, but another image area (that is, an image area that is originally unnecessary by being overwritten by the logo in FIG. 7A). ) Is drawn. In this case, it is necessary to overwrite the image area of the logo again. This is nothing more than a useless drawing and is a condition to avoid.

In order to avoid such a state, it is preferable to synchronize the timing of drawing the element to be acquired as an image. In the present embodiment, first, the elements necessary for printing are hidden, and control is incorporated to switch to the display setting after detecting the completion of loading of the elements to be acquired as an image. As described above, if the parent element is instructed to be hidden, the child element is also hidden. Therefore, a script and a hide setting are added to the SVG element which is the parent element. It should be noted that a mode may be adopted in which a hidden setting is added to each child element for control.

In this way, by changing the display setting at the timing when the resource reading and the DOM tree construction are all completed, it is possible to perform the synchronous drawing process of the element from the normal asynchronous drawing. As a result, it is possible to prevent the web engine 224 from executing the repetition of the drawing process of drawing from the drawable element. As a result, it is possible to avoid unnecessary load on the information processing device 115 such as a smartphone.

It should be noted that the change of the display setting is not limited to the above-mentioned display: block. Alternatively, it may be hidden by using an attribute such as visibility: hidden and displayed by visibility: visible. As another method, first, the x-coordinate and the y-coordinate of the SVG content are specified outside the display area. Then, it may be returned to the display area at the timing when onload can be detected. In other words, any command may be included as long as the web browser includes a command that causes the web browser to perform a behavior such as changing the element to the state of not displaying it on the display and then changing the element to the state of displaying it on the display. Even in this case, asynchronous drawing can be suppressed.

In S305_3, the script layer 217 transmits the modified SVG content to the native layer 218, and offscreen generation is performed in the native layer 218. When operating the SVG content with the print application 200, as described above, since it is in the form of DOM, it cannot be transmitted to the native layer 218 as it is. Therefore, it is once converted to the XML character string format. Since Javascript has a function to convert DOM into a character string, it may be used. For example, by using the "innerHTML" of the DOM or the serializeToString function of the XMLSerializer, data can be acquired from the DOM as an SVG character string (that is, SVG content). After performing this conversion, the script layer 217 transmits the SVG content converted into a character string to the native layer 218 by using the binding function.

In this embodiment, an example of modifying the SVG content in the script layer 217 has been described, but the SVG may be modified in the native layer 218. In the native layer, SVG of a character string is received, but parsing XML and operating it natively is an existing technology, and open source and the like also exist. Therefore, the editing of the DOM and SVG contents is not limited to the script layer 217, and the processing may be distributed among the native layers 218 as needed.

Subsequently, in S305_3, the native layer 218 prepares WebView for rendering. WebView is an area for drawing content written in a web language. Originally, a web browser is a tool for using WWW (World Wide Web), and is used for displaying contents written in a web language. Not only that, web browsers usually also have features for searching sites, page back buttons, or refresh buttons. On the other hand, WebView is an area for drawing content written in the Web language, and does not include auxiliary functions in browsing such as a back button. That is, WebView is an area that only draws the result in the Web language by using the function of the Web engine 224. Therefore, for example, by specifying the transparency, it is possible to realize an off-screen WebView that the image data can be acquired although it is not visible to the user (not displayed). In the present embodiment, the View used for taking a screenshot of the Web content in the native layer 218 is referred to as WebView.

In this example, the native layer 218 prepares a WebView of the same size as the SVG transmitted from the script layer 217. That is, a WebView of 4005px × 6000px including the size of the marker is prepared. This is because when the size of WebView is smaller than the size of SVG, it is possible to prevent the drawn content from protruding and an area where it is not drawn is generated. It should be noted that the processing may be performed for each small area by using the small WebView and the divided SVG suitable for the area. Performing processing in small areas at a time has the advantage that the memory load is reduced because the amount of RAM used is reduced. In this embodiment, for the sake of simplicity, the WebView size and the SVG are processed in the same size. In addition, WebView is placed in a position where it is not displayed, or is set to be transparent, so that it is invisible to the user, so-called off-screen, and processing is advanced so that the user cannot see it.

When the preparation of WebView is completed, the native layer 218 loads the SVG content transmitted from the script layer 217 into the prepared WebView, and proceeds to S305_4.

In S305_4, the native layer 218 acquires image data using the screenshot function of the OS. The screenshot function is a function for acquiring image data from WebView. Each OS has a function to acquire image data from the prepared View. View is an area for displaying some data on the display 104, and although the name is different depending on the OS, ImageView or WebView exists. For example, ImageView is an area for displaying a read standard format image (JPEG image, PNG image), and WebView is an area for displaying contents described in a web language. There are various types of data to be read in this way, but View defines an area for reading some data and displaying it on the display 104.

In S305_5, the native layer 218 determines an image to be used as a print image from the acquired image data. In this example, S305_2 is modified to synchronously draw SVG content and a marker is added. Therefore, if the marker exists in the image data acquired from WebView, it can be determined as the end of drawing. However, the marker itself is data unnecessary for printing. Therefore, on condition that the existence of the marker is confirmed, the native layer 218 trims only the area 4000px × 6000px necessary for printing, and determines this as image data. As described above, the image data determination of S305_5 is a step of confirming whether the acquired image data has necessary information and performing post-processing if predetermined post-processing is necessary. If the acquired image is an array of RGBA data and the image finally desired is in JPEG format, a process of converting to JPEG format may be performed. It should be noted that the above-mentioned trimming area may be defined in advance such that the right end 5px is an unnecessary area. In addition, the required area may be transmitted from the script layer 217 as an argument. Alternatively, a method such as treating only the area of the first child element of the modified SVG content as a necessary area may be used.

Here, if the presence of the marker cannot be detected in S305_5, the operation of S305_4 is repeated again. This repetition may be repeated immediately if the presence of the marker cannot be confirmed, or may be repeated after a certain period of time. Further, the timing of acquiring the image data of S305_5 may be started from any time point, or may be started from an arbitrary timing and confirmed at regular intervals.

Further, although the presence of the marker is used for determining the image data in the present embodiment, other methods may be used. For example, the image data may be acquired before the SVG is read, and the image data may be determined at the timing when the image change after the SVG is read is detected. Further, the content modification in the embodiment embeds a script that detects onload and sets the display in the SVG, but the display control of the SVG content is not limited to this example. For example, the native layer 218 may issue an instruction to WebView to operate the SVG content. It can be realized by preparing the following Javascript.
----------------------------------------------
[[Example of Javascript description]]
function () {
var svg = document. querySelector (“svg”) ； // Get svg
svg. style. display = “none” ； // Set to hide svg
setTimeout (function () {
svg. style. display = “block”; // Set to display svg after 1 second
callNativeMethod (); // Notify the native layer that the display settings have been made.
}, 1000);
}
----------------------------------------------

As described above, the display / non-display setting method is not limited to embedding the script in SVG. Any method can be used as long as the display / non-display of the elements required for imaging can be controlled.

In S305_6, the native layer 218 generates a print command. The generation of a print command is to generate data including an instruction to print the image data and information about the image data. An example is shown below.
----------------------------------------------
[[Example of print command]]
01: <? Xml version = "1.0" encoding = "utf-8"?>
02: <cmd xmlns: trans = "http: //www.xxxx/yyyyy/">
03: <contents>
04: <operation> Print </ operation>
05: <size> A4 </ size>
06: <quality> high </ quality>
07: <file>
08: <format> JPEG </ format>
09: <data> Image data body ...
</ Data>
… </ File>
</ Contents>
</ cmd>
----------------------------------------------

Print on the fourth line is an instruction to print to the printer. The fifth line indicates that the size is A4, and the sixth line indicates that the print quality is high and that printing is performed in the high quality mode. The 7th to 10th lines are information related to the image data. The eighth line shows that the image is in JPEG format. The image data body is stored in the 9th and subsequent lines.

In S305_7, the native layer 218 transmits the generated print command to the printer 112. In response to this, the printer 112 executes printing based on the given information.

As described above, according to the present embodiment, it is possible to prevent an unnecessary processing load from being generated when printing is performed using the hybrid application.

In this embodiment, a simple SVG content containing only a photo and a logo is assumed, but of course, three or more elements may be used. In the present embodiment, the switching of the display setting is described for all the elements, but it may be applied to only a part of the elements. That is, when the elements of the SVG content are composed of two or more elements, the process of switching the display settings in at least two elements may be performed. Even in such a case, unnecessary drawing instructions are reduced, which leads to an improvement in performance. Further, although the SVG content in which the logo is superimposed on the photograph has been described as an example, the content may be such that the elements are not superimposed.

The method of specifying the size of the image to be printed is not limited to the method shown in FIG. For example, the print application may display a screen for the user to input a numerical value such as "○○ mm × △△ mm" on the screen as the actual size of the image to be obtained by printing. The size input on such a screen is notified from the script layer 217 to the native layer 218. Then, the image processing unit 208 of the native layer 218 performs scaling and trimming of the image so as to fit the size. Then, the image obtained by scaling and trimming in this way is transmitted to the printer 112 via the printer communication unit 213 and printed. Therefore, an image of the size specified by the user on the above screen is printed.

This scaling and trimming may be performed on the SVG content in S305_1 or on the image data acquired in S305_1. Further, in the above example, the case where the print application is a hybrid application has been described, but the present invention is not limited to this, and the print application may be a native application.

<< Other Embodiments >>
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Claims (16)

The first layer consists of a set of script instructions that are translated and executed by the processor when the application is executed, and the second layer that is pre-compiled by a unit different from the processor as an instruction set that can be executed by the processor. A program having a layer and executing the application in cooperation with the first layer and the second layer.
The processor
The first generation means to generate printable content containing multiple elements,
The drawing means for asynchronously drawing the elements included in the content, the drawing means for drawing the at least two or more elements after at least two or more elements among the plurality of elements included in the content to be printed are loaded. A requesting means for requesting drawing of the content to be printed generated by the first generation means so that the means draws, and a requesting means for requesting drawing.
A second generation means that acquires an image drawn by the drawing means based on a request by the request means and generates data for printing.
A program to function as. The requesting means modifies the content to be printed generated by the first generation means so that the drawing means draws the at least two or more elements after the at least two or more elements are loaded. The program according to claim 1, wherein the drawing means is requested to draw the modified content. The requesting means is characterized in that the content to be printed is modified so that the at least two or more elements are hidden and the display is performed after the loading of the two or more elements is completed. The program according to claim 2. The requesting means displays the at least two or more elements in an area different from the display area, and displays the two or more elements in the display area after the loading of the two or more elements is completed. The program according to claim 2, wherein the content is modified. The program according to any one of claims 1 to 4, wherein the drawing means is an OS web engine. The program according to any one of claims 1 to 5, wherein the second generation means acquires the image by using the screenshot function of the OS. The program according to any one of claims 1 to 6, wherein the first layer is described in HTML or Javascript. The program according to any one of claims 1 to 7, wherein the content to be printed has at least one or more elements superimposed on at least a part of other elements. The plurality of elements include a predetermined marker, and the second generation means acquires the image on condition that the image drawn by the drawing means includes the predetermined marker. The program according to any one of items 1 to 8. The program according to any one of claims 1 to 9, wherein the second generation means acquires the image after all of the plurality of elements have been drawn by the drawing means. The program according to any one of claims 1 to 10, wherein the content to be printed is described using SVG. The program according to any one of claims 1 to 11, wherein the second generation means generates data for printing after performing a predetermined process on the generated image. A display control means that causes the display means to display a screen for the user to input a numerical value of the size of the image to be printed.
A scaling means for scaling the content to be printed or an image drawn by the drawing means according to a numerical value input on the screen displayed by the display control means.
The program according to any one of claims 1 to 12 for further functioning the processor. 13. Claim 13 for further functioning the processor as a trimming means for performing trimming on the content to be printed or the image drawn by the drawing means according to the numerical value input on the screen displayed by the display means. The program described in. With the processor
A first layer composed of a script instruction set translated and executed by the processor when the application is executed, and a first layer pre-compiled by a unit different from the processor as an instruction set that can be executed by the processor. A storage medium having two layers and storing a program for executing the application in cooperation with the first layer and the second layer.
It is an information processing device equipped with
When the processor executes the program stored in the storage medium, the processor can be
The first generation means to generate printable content containing multiple elements,
The drawing means for asynchronously drawing the elements included in the content, the drawing means for drawing the at least two or more elements after at least two or more elements among the plurality of elements included in the content to be printed are loaded. A requesting means for requesting drawing of the content to be printed generated by the first generation means and a requesting means for drawing the content to be printed so that the means draws.
A second generation means that acquires an image drawn by the drawing means based on a request by the request means and generates data for printing.
An information processing device characterized by functioning as. The first layer consists of a set of script instructions that are translated and executed by the processor when the application is executed, and the second layer that is pre-compiled by a unit different from the processor as an instruction set that can be executed by the processor. It is an information processing method performed by executing the application in cooperation with the layer.
The first generation step to generate printable content containing multiple elements,
The drawing means for asynchronously drawing the elements included in the content, the drawing means for drawing the at least two or more elements after at least two or more elements among the plurality of elements included in the content to be printed are loaded. A request step for requesting drawing of the content to be printed generated in the first generation step so that the means draws, and a request step.
A second generation step of acquiring an image drawn by the drawing means based on a request by the request step and generating data for printing, and a second generation step.
An information processing method characterized by having.

Images