JP6757368B2 - Information processing equipment, processing methods, and programs - Google Patents

Description

The present invention relates to, for example, a rendering process executed by an information processing device such as a mobile terminal that operates as an external device for an image output device such as a printer.

In recent years, portable multifunctional mobile terminals (hereinafter, mobile computers) equipped with camera functions have exploded in popularity, and sales have expanded at a rate that far exceeds the number of digital cameras and conventional personal computers (hereinafter, PCs) sold. ing.

Such a mobile computer basically consists of three elements. That is, the hardware that is the computer itself, the operating system (hereinafter, OS) that runs on the hardware, and the application that runs on the OS. The user can use the application to use functions such as maps, e-mails, and browsing websites on the Internet.

There are two main forms of applications that run on such mobile computers. That is, native applications and web applications. The features of each will be described below.

First, a native application is usually developed using a development environment and a development language prepared for each OS. For example, the C / C ++ language is used on the OS provided by the company A, the Java (registered trademark) language is used on the OS provided by the company B, and a different development language is used on the OS provided by the company C. Normally, a native application is precompiled in each development environment and converted from a so-called high-level language that can be understood by humans to a set of instructions such as an assembler that can be interpreted by a computer CPU. As described above, in a normal native application, since the CPU directly interprets the instruction, there is an advantage that high-speed operation is possible.

On the other hand, a web application is an application that runs on a web browser that is standardly embedded in an OS on each computer in recent years. The application is generally developed using languages such as HTML5 and CSS, as well as JavaScript® so that it can be interpreted by a web browser. Since these are web standard languages, once a web application is written in these web standard languages, there is an advantage that it can be operated in any environment where a web browser operates.

Japanese Unexamined Patent Publication No. 2011-23304

The recent mobile computers mentioned above have a built-in high resolution camera. Since mobile computers are carried by users on a daily basis and have a memory that can store thousands of photographs, users can enjoy taking photographs very frequently and comfortably. The user can perform image processing on the photographic image obtained in this manner, for example, by applying a filter process to make a monochrome sepia tone, or to correct problems such as a dark photograph or an imbalance in color. It is a very important and indispensable function for us. When developing an application in the above two forms in which a user can easily execute such image processing without stress, there are the following problems.

First, native applications have the advantage of being able to execute processing at high speed as described above. However, since this native application needs to be developed separately in a different development language for each OS, there is a problem that the development cost and the development time increase and it cannot be provided to the user quickly. In addition, the native application needs to be compiled (translated) in advance. Therefore, for example, it is difficult to change the UI design of the application at the time of operation or dynamically add functions, and it lacks flexibility.

Patent Document 1 discloses an example of a form of a web application. In a web application, the main body of the application described in HTML5, CSS, and Javascript (registered trademark) usually exists on a server outside the mobile computer. Since the application is dynamically downloaded from the server to the mobile computer via the Internet at the time of use, it is possible to dynamically change the UI design without compiling in advance. However, when executing sophisticated and complicated processing, the web application has only two options, that is, it is executed by Javascript (registered trademark) on the browser or it is executed on the server due to the security limitation of the browser. Javascript (registered trademark) has conventionally been described as a human-visible character string script, and the script can be executed by compiling the script at any time during operation. For this reason, if a complicated process is described in Javascript (registered trademark), there is a problem that the operation is slow.

Also, if the process is built to be executed on the server, it will take time to upload the data such as photos existing inside the mobile computer to the server via the Internet and download the processed result this time. Become. This is a major problem for users who want less stressful, immediate processing of their mobile applications. In addition, there is a problem that the processing on the server cannot be executed offline.

Further, when the image enlargement processing is executed in the image processing, it is necessary to hold the values of the pixels before and after the enlargement in the memory. Therefore, the larger the number of pixels, the more the memory is used. However, when a mobile computer, for example, executes an application as described above, it is difficult to implement a large-capacity RAM as a memory. In addition, since the web application as described above holds the image data as character string data instead of binary data, a large amount of memory is required as compared with the case where the same amount of image data is held as binary data. Therefore, the image enlargement process requires a larger amount of memory, and the computer tends to run out of memory.

The present invention has been made to solve the above problems, and is an information processing device and a processing method capable of efficiently processing image data without causing a memory shortage even when executing a hybrid application. , And the purpose of providing the program.

The information processing apparatus of the present invention for achieving the above object has the following configuration.

That is, a hybrid application including a first program layer in which an instruction set to be translated and executed by a processor is described, and a second program layer in which an instruction set pre-translated by a processor other than the processor is described. A plurality of information processing devices that execute a program on the processor, obtained by a division / enlargement means that executes division processing and enlargement processing on image data, and a plurality of division processing and enlargement processing by the division / enlargement means. It has an output means for executing output processing to an external device using image data, and the instruction set of the first program layer includes any one of HTML, CSS, and Javascript, and the instruction set of the first program layer includes the above. The division / enlargement means sets the position information of the image data and the size information of the image data using a Web drawing language, and executes the division processing and the enlargement processing on the image data based on the position information and the size information. It is characterized by.

Looking at the present invention from another aspect, a first program layer in which an instruction set to be translated and executed by a processor is described, and a first program layer in which an instruction set pre-translated by a processor other than the processor is described. A processing method of an information processing apparatus that executes a hybrid application program including two program layers by the processor, in which division processing and enlargement processing are executed on image data in the first program layer, and the division processing is performed. Output processing to an external device is executed using the plurality of image data obtained by the processing and the enlargement processing, and the instruction set of the first program layer includes any one of HTML, CSS, and Javascript, and is drawn on the Web. A process characterized by setting position information of image data and size information of image data using a language, and executing the division process and the enlargement process on the image data based on the position information and the size information. Provide a method.

Looking at the present invention from another aspect, a hybrid program including a first program layer written in a web standard language and a second program layer written in a programming language different from the web standard language is executed. A computer executes output processing to an external device using a division / enlargement means that executes division processing and enlargement processing on image data and a plurality of image data obtained by the division processing and enlargement processing by the division / enlargement means. The instruction set of the first program layer includes any one of HTML, CSS, and Javascript, and the division / enlargement means of the first program layer uses a Web drawing language to generate image data. A computer-readable program that sets the position information and the size information of the image data, and executes the division process and the enlargement process on the image data based on the position information and the size information. To be equipped.

Therefore, according to the present invention, there is an effect that the occurrence of memory shortage can be suppressed even when performing image processing using a large capacity memory such as image enlargement processing.

It is a block diagram which shows the structure of the information processing apparatus which is a typical embodiment of this invention. It is a block diagram which shows the software structure of the information processing apparatus shown in FIG. It is a flowchart which shows the process which accompanies the user operation. It is a flowchart which shows the detail of the photographic image selection according to Embodiment 1. It is a flowchart which shows the detail of the image processing according to Embodiment 1. It is a flowchart which shows the detail of the stamp addition according to Embodiment 1. It is a flowchart which shows the detail of a stamp specific. It is a flowchart which shows the detail of a stamp operation. It is a flowchart which shows the details of a printer setting. It is a flowchart which shows the detail of the rendering according to Embodiment 1. It is a flowchart which shows the detail of a print. It is a figure which shows an example of an application screen. It is a figure which shows an example of the setting screen. It is a schematic diagram which conceptually shows a band rendering. It is a flowchart which shows the detail of the band rendering according to Embodiment 1. It is a flowchart which shows the detail of the rendering according to Embodiment 2. It is a flowchart which shows the detail of the band rendering according to Embodiment 2.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In particular, in the following embodiment, a configuration in which a hybrid photo printing application described later is operated on an information processing device, various image processes are applied to an image selected by a user, and then the image is printed. explain. A typical example of the information processing device includes a portable information terminal such as a mobile phone, a smartphone, or a tablet terminal.

<Explanation of hardware configuration>
FIG. 1 is a block diagram illustrating a configuration example of a portable information terminal such as a smartphone or a mobile phone as the information processing device 115 which is a typical embodiment of the present invention. In the figure, 100 is a CPU (Central Processing Unit / Processor), and executes various processes described below according to a program. Although the number of CPUs 100 in the figure is one, it may be composed of a plurality of CPUs or CPU cores. Reference numeral 101 denotes a ROM, which stores a program executed by the CPU 100. Reference numeral 102 denotes a RAM, which is a memory for temporarily storing various information when the program is executed by the CPU 100. Although one example of the CPU is shown in FIG. 1, a configuration using a plurality of CPUs, that is, a multiprocessor configuration may be used.

Reference numeral 103 denotes a secondary storage device such as a hard disk or a flash memory, which is a storage medium for storing data such as a database that holds processing results such as files and image analysis, and various programs. Reference numeral 104 denotes a display, which displays various information such as a UI (user interface) for receiving operations for realizing various processes and processing results of the executed processes. The display 104 may include a touch sensor 105.

The information processing device 115 may include an internal imaging device 110. The image data obtained by imaging by the internal imaging device 110 is stored in the secondary storage device 103 after undergoing predetermined image processing. The image data can also be read from the external imaging device 111 connected via the external I / F (interface) 108.

The information processing device 115 includes an external I / F (interface) 109, and can communicate via a network 113 such as the Internet. The information processing device 115 can also acquire image data from the server 114 connected to the network 113 via the communication I / F 109.

The information processing device 115 includes an acceleration sensor 106, and can acquire acceleration information regarding the position and orientation of the information processing device 115 itself. The information processing device 115 is connected to the printer 112 via an external I / F 107, and can output data such as image data. The printer 112 is also connected to the network 113, and can transmit and receive image data via the communication I / F 109.

The external I / Fs 107 to 109 are interfaces having at least one of wired communication and wireless communication, and communicate with an external device (printer 112 or server 114) according to the communication mode to be used. Wired communication includes, for example, USB, Ethernet (registered trademark), and the like, and wireless communication includes wireless LAN, NFC, Bluetooth (registered trademark), infrared communication, and the like. Further, when a wireless LAN is used as wireless communication, there are a form in which the devices are directly connected to each other, and a form in which the devices are connected via a relay device such as a wireless LAN router. Further, although the external I / Fs 107 to 109 are separately configured in the figure, they may be integrally configured.

The power supply required for the operation of the information processing apparatus 115 is supplied by the battery 117. The various components included in the information processing apparatus 115 are connected to each other via the control bus / data bus 116, and the CPU 100 controls the various components via the control bus / data bus 116.

In the present embodiment, the information processing device 115 serves as an execution place (software execution environment) for software such as a program executed by the control unit (CPU 100) included in the information processing device 115.

<Software block diagram>
FIG. 2 is a block diagram of a software configuration operated by the information processing device 115.

The information processing device 115 executes the programs of the script layer 217, the native layer 218, and the OS layer 219. Each of these layers is realized by the CPU 100 reading and executing the corresponding program stored in the ROM 101 or the secondary storage device 103.

The script layer 217 is a program layer in which an instruction set (drawing of contents, display of images, reproduction of moving images, etc.) is described by text data using a web standard language such as HTML5, CSS3, and Javascript. In the script layer 217, various instruction sets of text data are translated and executed on the application execution environment by using a processor (for example, CPU 100) existing in the application execution environment. As the execution form, there are cases where the statement is dynamically translated line by line each time it is executed, when the application is started, and when the application is installed on the information processing device 115. Be done.

Hereinafter, the processing and contents of the script layer 217 will be referred to as a script. As an example of the form of translating the script instruction in the information processing apparatus 115, it is possible to use the interpreter function provided in the native layer 218 and the OS layer 219. In this embodiment, it is assumed that most of the UI of the application is described by the script layer 217.

The native layer 218 is a program layer in which an instruction set pre-translated (compiled) other than the application execution environment is described. As a form, code written in a high-level language such as C or C ++ is compiled in advance on the PC or server of the application developer, and is a set of instructions that can be interpreted by the CPU 100. Hereinafter, the processing and contents of the native layer 218 and the functions of the OS layer 219 described later will be referred to as native, including calling from the native layer 218. As another mounting system of the native layer 218, Java can be mentioned. Java is a high-level language similar to C / C ++, and is translated into intermediate code in advance in the development environment at the time of application development. The translated intermediate code operates in the Java virtual environment provided by each OS. In the present embodiment, such a program form is also included in a kind of native layer 218.

The OS layer 219 corresponds to the operating system (OS) of the information processing device 115. The OS layer 219 has a role and a unique function of providing the application with the use of the hardware function. The OS layer 219 includes an API, and functions can be used from the script layer 217 and the native layer 218.

In this embodiment, enabling the call of the native layer 218 from the script layer 217 is called binding or binding. Various native functions are provided with APIs, and native functions can be used by calling the APIs by a script. Such a binding function is usually a function that various OSs have as standard.

In this embodiment, the application including the script layer 217 and the native layer 218 is referred to as a hybrid application.

The image acquisition unit 201 of the script layer 217 requests the native layer 218 to acquire image data. At the time of the acquisition request, the image acquisition unit 201 generates a unique ID and transmits it to the native layer 218. This ID and the image data read by the image reading unit 202 of the native layer 218 are stored as a pair in the data holding unit 204 of the native layer 218. In addition to this, for example, a method of specifying an absolute path, a method of prompting a dialog display, and the like can be mentioned as a method of requesting acquisition.

The image reading unit 202 of the native layer 218 acquires image data from the image data group 215. The method of acquiring the image data from the image data group 215 depends on the request of the image acquisition unit 201 of the script layer 217. The request method includes selecting from the dialog box provided on the UI, selecting the image directly from the file path, and the like.

The data conversion unit 203 of the native layer 218 converts the data of the native layer 218 (example: image data in binary format) into data in a format that can be used in the script layer 217 (example: image data in text format (BASE64)). On the other hand, the data conversion unit 203 converts the data sent from the script layer 217 (example: image data in text format (BASE64)) into data in a format that can be used in the native layer 218 (example: image data in binary format). It also performs conversion.

The data conversion unit 207 of the script layer 217 converts the data of the script layer 217 (example: text format processing parameters) into the data of the format that can be used by the native layer 218 (example: text format (JSON format) processing parameters). ..

The data holding unit 204 of the native layer 218 holds the image data read by the image reading unit 202 and the image data processed by the image processing unit 208. The retained image data is expanded into, for example, RGB image data, and is in a format in which image processing can be immediately executed. Further, the retained image data is paired with the ID generated by the image acquisition unit 201, and the ID may be specified when acquiring the image data from the data acquisition unit 204.

The content drawing unit 205 of the script layer 217 describes the content for printing by using a web standard language. The script operated by the content operation unit 210 is also reflected in this description. The content script described by the content drawing unit 205 is interpreted by the interpreter 214 of the OS layer 219 and displayed on the display 104.

The image processing control unit 206 of the script layer 217 determines the correction parameters used for image processing and the image to be processed, and requests the image processing unit 208 to perform image processing. The correction parameter is converted into a format that can be transmitted to the native layer by the data conversion unit 207, and then transmitted to the native layer together with the ID of the image to be processed.

The image processing unit 208 of the native layer performs image processing on the image specified by the image processing control unit 206. At that time, what kind of image processing is performed is determined by the parameters set by the image processing control unit 206. Regarding the designation of the image, for example, a method of receiving the image path from the script layer, a method of receiving the entire image data, and the like can be considered.

The touch event 209 of the OS layer 219 acquires information regarding the touch of the display 104. Examples of the touch-related information include touch detection of the display 104, touched position information, and the like. The acquired information regarding the touch is transmitted to the content operation unit 210 of the script layer 217 via the native layer 218.

The content operation unit 210 of the script layer 217 operates the image, for example, enlarges, moves, or rotates the image, and changes the script command in order to reflect the operation.

The printer control unit 211 of the script layer 217 controls a rendering start request, a printer detection request, a printer setting screen display, and print information generation and transmission to the rendering unit 216. On the printer setting screen, printer settings such as paper size, paper type, and color / monochrome are made. The printer data is generated by the printer data generation unit 212 based on the items set here.

The printer data generation unit 212 of the native layer 218 generates data and commands necessary for printer communication based on a request from the printer control unit 211. The data required for printer communication is data according to the communication protocol, and the command is data for determining the operation of the printer such as printing and scanning. Therefore, the printer data generation unit 212 generates printer data including commands for determining the operation of the printer.

By the way, the external device communication unit 221 of the native layer 218 is an interface (IF) for communicating with an external device connected to the information processing device 115 such as a printer. Here, the data received from the printer data generation unit 212 is transmitted, and the information from the printer 112 is received. In this embodiment, communication is performed via the communication control unit 213 of the OS layer 219, but the external device communication unit 221 may directly transmit data to the external IF 107. If the communication control unit 213 of the OS layer 219 supports the communication protocol used by the external device, the function may be used, and if the communication control unit 213 does not support the communication protocol used by the external device, the external device communication unit 221 Communicates according to its communication protocol.

The interpreter 214 of the OS layer 219 interprets and executes the instruction written in the web standard language generated by the script layer 217. For example, commands such as drawing an image are executed through the interpreter 214 and displayed on the display 104.

The image data group 215 is an area that holds image data. The data storage unit 220 functions to store the image data held by the data storage unit 204 in the image data group 215, if necessary.

The rendering unit 216 controls the content drawing unit 205, the image processing control unit 206, and the content operation unit 210 to render the image data to be processed. This rendering includes, for example, generating an image at the output resolution to the printer 112 at the script layer 217. Further, the rendering result in the script layer 217 and the image in which the script layer 217 is being generated are not displayed on the display 104. The rendering result is transmitted to the data conversion unit 203 of the native layer 218 and converted into image data in a format that can be used by the printer 112.

<Processing associated with user operation>
FIG. 3 is a flowchart showing a process including a user operation. First, the outline of each process of S21 to S28 will be described with reference to FIG. 3, and the details will be described later. The processing of each step of the flowchart of the present application is realized by the CPU 100 of the information processing device 115 executing the program stored in the ROM 101 or the secondary storage device 103. Further, each step shown in FIG. 3 transitions according to a user operation on the application screen 1200 shown in FIG. 12, which is one of the UIs. The application screen 1200 is generated by the script layer 217. The operation of the application screen 1200 is realized, for example, via the touch sensor 105.

In S21, when the CPU 100 detects a user operation (including touch operation input; the same applies hereinafter) for the photo image selection button 1201 on the application screen 1200, the CPU 100 selects an arbitrary image according to the operation. When an image is selected, the CPU 100 displays the selected image in the drawing area 1206 of the application screen 1200.

In S22, when the CPU 100 detects a user operation on the slide bar 1202 for adjusting the brightness of the displayed image, the CPU 100 sets a correction parameter to be used during image processing according to the user operation. Then, the CPU 100 performs image processing on the displayed image according to the set correction parameter, and displays the processing content and the processing result in the drawing area 1206.

In S23, when the CPU 100 detects a user operation on the stamp addition button 1203, the CPU 100 displays a stamp list. When the selection of the stamp is detected by the user operation on the stamp list, the CPU 100 adds / displays the selected stamp to the drawing area 1206.

In S24, the CPU 100 identifies the stamp in response to a user operation on the application screen 1200. The identification of the stamp is to determine whether or not the stamp has been touched from the coordinates touched by the user operation on the display 104 and the coordinates of the stamp. When the stamp is touched, the stamp is in the operation acceptance state. Here, it is assumed that the stamp is in the operation acceptance state according to the user operation. The operation acceptance status will be described later.

In S25, when the CPU 100 detects a user operation of swiping the stamp in the display area in the operation acceptance state, the stamp in the operation acceptance state moves in the drawing area in response to the user operation.

When the CPU 100 detects a user operation on the print button 1205 in S26, the CPU 100 displays a setting screen 1301 (FIG. 13) for setting information necessary for printing. The information required for printing includes, for example, as shown in the setting screen 1301 of FIG. 13, setting items of paper size, paper type, print quality, and with / without border. In addition to this, setable setting items such as double-sided / single-sided, monochrome / color, etc. are configured according to the functions of the printer to be used.

In S27, when the CPU 100 detects a user operation on the setting completion button 1302 on the setting screen 1301, the CPU 100 executes rendering for converting the image displayed in the drawing area 1206 into a print resolution for output to the printer.

In S28, the CPU 100 transmits the image converted to the resolution of the printer to the printer 112 together with the printer control command. By the above processing, the image selected by the user is printed by the printer 112.

The processing shown in FIG. 3 is an example, and the processing content is not limited to this, and the processing order of the step group is not limited to this. Further, in this embodiment, the first program layer including the instruction set to be translated and executed by the processor is the script layer 217, and the second program layer including the instruction set pre-translated by a non-processor is the native layer 218. Is defined as. Then, a program including these first program layer and the second program layer realizes a hybrid application. Character string data (text data) is defined as the first format, and binary data is defined as the second format. The script layer 217 can hold text format data, and the native layer 218 can hold binary format data.

<Details of photographic image selection and its image processing>
Next, some embodiments relating to photographic image selection and image processing associated therewith to be executed in the information processing apparatus having the above configuration will be described. Therefore, in these cases, the information processing device 115 functions as an image processing device.

[Embodiment 1]
S21 is started when the user presses the photographic image selection button 1201 shown in FIG. Here, the details of the photographic image selection in S21 of FIG. 3 will be described with reference to FIG. Note that S301 to S302 and S309 to S311 are processes executed by the CPU 100 using the program of the script layer 217, and S303 to S308 are processes executed by the CPU 100 using the program of the native layer 218.

In S301, the CPU 100 generates a unique ID. This ID may be in any form such as a numerical value or a character string as long as it can be transmitted from the script layer 217 to the native layer 218. In S302, the CPU 100 requests the native layer 218 to select an image in response to a user operation on the photographic image selection button 1201. In S302, the ID generated in S301 is also transmitted to the native layer 218. In that request, the binding function directly calls the image selection API specific to the native layer 218 from the script layer 217. However, if the image selection API specific to the native layer cannot be called directly, prepare a function that can be called directly from the script layer 217 in the native layer 218, and describe a function that calls the image selection API specific to the native layer in the function. Just leave it. This is a method of preparing a trumpet in advance. Further, the API includes a mechanism for passing the ID as an argument, for example. In this way, the ID is passed to the native layer 218.

In S303, the CPU 100 displays the device-specific image selection UI on the display 104. Select one arbitrary image based on the user operation for the displayed image selection UI. In this embodiment, one image is selected from the image folder in the device, but the present invention is not limited thereto. For example, an image on the Internet or an image in a removable storage medium may be selected, or an image may be taken on the spot using the camera function of the device and selected.

In S304, the CPU 100 acquires the selected image. For example, if the selected image is in the state of an image file, the CPU 100 opens the file and reads its contents.

In S305, the CPU 100 develops the acquired image in RGB space. Here, the RGB data is stored in the native layer 218, but the present invention is not limited to this. For example, it can be held in JPEG (Joint Photographic Expert Group), PNG (Portable Network Graphics), RGBA format, or the like. The RGBA format is a combination of RGB (red, green, blue) components of image data and A as transparency.

In S306, the CPU 100 holds the developed RGB image in the data holding unit 204 in association with the ID acquired from the script layer 217. As an association method, for example, a method of creating an object having an ID and an RGB image so that the RGB image can be specified by the ID can be considered. This association method is not limited to this, and a path that is an access destination of the ID and the selected image, a function or a class that is executed according to the ID and RGB expansion, and the like can be considered.

In S307, the CPU 100 converts the expanded RGB image into image data in a format (supportable format) that can be used by the script layer 217. In this embodiment, the data format to be converted is JPEG (Joint Photography Expert Group). The conversion from the RGB image data to the JPEG data may be performed by using the encoder provided in the OS layer 219.

Here, if the image is too large for the display size of the display 104, it is reduced and then converted to JPEG. The script will reduce the image according to the size of the UI and the resolution of the display. If an image with an excessive number of pixels for the UI is handled by a script, a processing load is applied to the CPU and a large amount of memory is required, which causes waste. Therefore, in this embodiment, the reduction process is executed in order to avoid the waste. As a guideline for the reduction process, for example, the display size of the display or the size of the part where the image is displayed on the UI may be set as the upper limit, and the reduction process may be controlled for the image having the number of pixels exceeding the upper limit. .. By the way, the image may be subjected to image processing as described later. Also for this, image processing is performed on the original image (image data developed in RGB), and the reduced image is passed to the script layer after the above reduction processing is executed.

In S308, the CPU 100 converts the JPEG format data into BASE64 data and transmits it to the script layer 217. This is because the RGB image data array cannot be used as it is in the script layer 217, and therefore it is necessary to convert it into a format that can be interpreted by the script layer 217 in the native layer 218. Since Javascript (registered trademark) can handle only character strings, in this embodiment, the format of BASE64 that expresses data as a character string is used.

Here, BASE64 is an encoding method for handling binary data as character string data. Similar to the native layer 218, the script layer 217 can acquire image data and make changes. However, the native layer 218 holds the value in a continuous area of the RAM 102 (memory) like the RGB array, whereas the script layer 217 does not always hold the data in the continuous area. As described above, since the data holding method is different between the script layer 217 and the native layer 218, it has not been possible to collectively transmit the image data from the script layer 217 to the native layer 218 as continuous data.

However, since the character string data is also treated as continuous data in the script layer 217, the CPU 100 transmits the continuous data from the script layer 217 to the native layer 218 in the BASE64 format. In this embodiment, the native layer 218 performs image information conversion processing. That is, by converting the character string information received from the script layer 217 into image data, data manipulation in the native layer 218 is possible. On the contrary, when the image data is transmitted from the native layer 218 to the script layer 217, the image data handled by the native layer 218 is converted into character string information by using the BASE64 method. As a result, image data can be transferred between the script layer 217 and the native layer 218.

In S309, the CPU 100 receives the BASE64 data converted by the native layer 218, and secures a drawing area for displaying the BASE64 data in the RAM 102. In this embodiment, the HTML function of Cambas is used as an example of securing the drawing area, and the API of the Context object possessed by Cambas is used for drawing the image.

In S310, the CPU 100 generates a correction parameter and initializes it. Here, the correction parameter is an object that holds a parameter group that determines the content of the image processing of S22. What kind of image processing is performed in the native layer is determined by the correction parameters. As an example of the correction parameters held by Javascript (registered trademark), the following forms can be considered.

var CorrectionParam = function () {
this.brightness = 10;
}
This correction parameter indicates that a variable named brightness is provided for brightness correction in the DirectionParam object, and a value of 10 is stored.

In this embodiment, for simplification of the description, the correction parameters are only for brightness (luminance) correction, but other parameters for correction processing (blur filter strength, sepia conversion on / off, etc.) are set. Needless to say, it is possible to add more types of processing.

In S311 the CPU 100 specifies BASE64 data as the data to be drawn in the drawing area, and draws an image in the drawing area accordingly. Specifically, the CPU 100 transmits the BASE64 data specified in the script layer to the interpreter 214 in the OS layer. Then, the interpreter 214 interprets the script of the BASE64 data and displays it as an image in the drawing area. Here, an example of sample code that reflects BASE64 data in the drawing area is shown.

--------------------------------------------------
var base64Data = BASE64 data from the native layer
var canvas = document.createElement ("canvas"); // Secure drawing area for image
canvas.setAttribute ("width", 100); // Set the size of the drawing area
canvas.setAttribute ("height", 100);
var context = canvas.getContext ("2d"); // Generate an object with API to draw in the drawing area
var img = new Image (); // Create Image object
img.src = base64Data; // Let the URI of the image be the received BASE64 data
img.onload = function () {// Start processing after the image has finished loading
context.drawImage (img, 0, 0, img.width, img.height, 0, 0, canvas.width,
canvas.height); // Draw the image in the drawing area using the method of the context object
document.getElementById ("div"). appendChild (canvas);
// This flowchart assumes a layered structure with multiple layers of Canvas}
These Cambas do not exist freely everywhere, and operations such as drawing, moving, and enlarging specify an area that is completed within a specific area (drawing area 1206 in FIG. 12). It is a "div", and Canvas takes the form of being added to the div.

--------------------------------------------------
In this embodiment, a layer structure in which Canvas is added to the drawing area 1206 is used. Since Canvas is treated as one image, if a stamp or the like is added after drawing a photographic image on Canvas, the stamp is also combined into one image. In this case, it is difficult to specify and operate only the stamp in the image. On the other hand, if Canvas is displayed in layers using a layer structure, it looks like a single picture to the user, but the actual drawings are independent of each other, and each is instructed to perform an individual operation. Can be done. Since the stamp is added to the image data, it may be called an additional image.

In this way, the image and the stamp are displayed on the UI as a single picture (image). In this embodiment, the image is the print content. However, the image at this point is displayed on the UI for operation and confirmation. As will be described later, in order to print, it is necessary to convert to a resolution suitable for printing.

<Details of image processing>
S22 is started when the user changes the slide bar 1202 shown in FIG. Here, the details of the image processing of S22 of FIG. 3 will be described with reference to FIG. Note that S401 to S403, S409, and S411 are processes executed by the CPU 100 using the program of the script layer 217, and S404 to S408 and S410 are processes executed by the CPU 100 using the program of the native layer 218.

In S401, the CPU 100 sets a correction parameter. Here, the value of the correction parameter brightness generated in S310 of FIG. 3 is set to a value set according to the user operation on the slide bar 1202. In S402, the CPU 100 activates the indicator and displays it on the display 104. Here, the indicator is a display that informs the user that processing is in progress, and is generally represented by an image such as a progress bar, a clock mark, or a blinking or rotating figure. In S403, the CPU 100 converts the correction parameter set in S401 into a JSON character string format that can be used in the native layer 218. Here, since the correction parameter is in the form of an object and cannot be used as it is in the native layer 218, the set correction parameter is converted into a JSON character string. Then, the CPU 100 transfers the correction parameter converted into the JSON character string to the native layer 218 together with the ID generated in S301 of FIG.

In S404, the CPU 100 decodes the correction parameter converted into the JSON character string and acquires the correction parameter. More specifically, the correction parameter is parsed (analyzed) by using the parser provided in the OS layer 219. After parsing, in the case of the above example, the brightness in the correction parameter will be acquired.

In S405, the CPU 100 identifies the RGB image developed in S305 of FIG. 4 based on the ID acquired from the script layer 217. As described above, the association between the ID and the image is not limited to the pairing of the ID and the RGB image, and for example, a method of associating the ID and the image path may be used. In addition, as an example of associating with the ID, various things such as an object of the native layer 218, a start address of image data, and a function for calling an image can be considered.

In S406, the CPU 100 determines the image processing to be performed from the acquired correction parameters, and performs the image processing on the RGB image. In this embodiment, 10 is added to the RGB values of all the pixels from the brightness correction parameter. As described above, other information may be added to the correction parameters to increase the types of image processing. For example, known monochrome conversion, known sepia color conversion, "ImageFix", "RedeeyeFix", "SmartSkin" and the like can be added.

Here, "ImageFix" is a function (face detection function) that automatically analyzes a photographic image using a human face detection or a scene analysis unit and adjusts appropriate brightness and white balance (Japanese Patent Laid-Open No. 2010). -278708 (see). Further, "Red-eye Fix" is a function (red-eye detection function) that automatically detects and corrects a red-eye image from an image (see JP-A-2006-350557). Further, "SmartSkin" is a function of detecting a person's face from a photographic image and appropriately processing the skin region of the face (see JP-A-2010-010938). The type of image processing function is not limited to this, and various image processing can be used depending on the application and purpose. Further, the image processing may utilize the function provided by the OS layer 219, and even in that case, the processing configuration is the same as that of this embodiment.

In S407, the CPU 100 converts the image-processed RGB image into image data in a format (supportable format) that can be used by the script layer 217. Here, the data is converted into JPEG format data in the same manner as in S307 of FIG. If the image size is larger than the UI as described above, the reduction process is executed before converting to JPEG data. In S408, the CPU 100 requests the script layer 217 to stop the indicator. This is achieved by calling the indicator stop function defined in the script layer 217 from the native layer 218.

In S409, the CPU 100 stops the indicator and erases the indicator from the display 104.

On the other hand, in S410, the CPU 100 converts the converted JPEG format data into BASE64 data and transmits it to the script layer 217.

In S411, the CPU 100 receives the BASE64 data converted in the native layer 218, and draws an image in the drawing area secured in S309 of FIG. 4 accordingly. Specifically, the CPU 100 transmits the BASE64 data specified in the script layer to the interpreter 214 in the OS layer. Then, the interpreter 214 interprets the BASE64 data script and displays the drawing result of the image data in the drawing area of the designated UI. By the above processing, the image data to which the image processing based on the correction parameter is applied is displayed.

In this embodiment, the image processing is started by the change of the slide bar 1202 as shown in FIG. 12, but the starting method is not limited to this. For example, a plus button and a minus button may be arranged on the screen, and the brightness may be adjusted each time the button is pressed. In addition, it may be realized by processing linked with a touch event, such as increasing the brightness when the right half of the image is touched and darkening when the left half is touched. Further, it is also possible to adopt a method in which only the correction parameters are changed by the user operation and all the image processing is performed at once when the execution instruction of the image processing is given.

<Details of adding stamps>
When the user presses the stamp addition button 1203 shown in FIG. 12 and selects the heart stamp 1208, the process of S23 starts. Here, the details of adding the stamp of S23 of FIG. 3 will be described with reference to FIG. In the following description, a case where the heart stamp 1208 is selected after the stamp addition button 1203 on the application screen 1200 of FIG. 12 is pressed and the stamp list is displayed by the user operation will be given as an example. Note that S501 to S502 and S508 to S510 are processes executed by the CPU 100 using the program of the script layer 217, and S503 to S507 are processes executed by the CPU 100 using the program of the native layer 218.

In S501, the CPU 100 generates a unique ID. Since this ID is equivalent to the ID generated in S301 of FIG. 3, it is generated according to the method described in S301. In S502, the CPU 100 requests the image selection of the stamp image corresponding to the stamp by transmitting the access destination (absolute path) of the image used as the stamp to the native layer 218 together with the ID generated in S501. ..

In S503, the CPU 100 acquires the stamp image by using the absolute path of the stamp image received from the script layer 217 and the device-specific image selection API. In S504, the CPU 100 develops the acquired stamp image in RGB. In S505, the CPU 100 holds the expanded RGB image in the data holding unit 204 in association with the ID acquired from the script layer 217. The association method is the same as S306 in FIG. In S506, the CPU 100 converts the expanded RGB image in the native layer 218 into image data in a format (supportable format) that can be used in the script layer 217. The conversion here is converted to JPEG format data in the same manner as in S307 of FIG. In S507, the CPU 100 converts the JPEG format data into BASE64 data and transmits the data to the script layer 217.

In S508, the CPU 100 receives the BASE64 data converted by the native layer 218, and secures a drawing area for displaying the BASE64 data in the RAM 102. In S509, the CPU 100 generates and initializes the object parameters. Here, the object parameter is an object that holds a parameter used for determining the rotation angle of the stamp after rendering at the time of rendering (details will be described later) of S27 of FIG. As an example of the object parameter held by Javascript (registered trademark), the following form can be considered.

var ObjectParam = function () {
this.theta = 0;
this.posX = 0;
this.posY = 0;
this.width = 100;
this.height = 100;
}
This object parameter indicates that the variable name "theta" indicating the rotation angle and the value "0" are stored in the ObjectParam object. Similarly, posX represents the x-coordinate when the upper left of the drawing area (1206 in FIG. 12) is the reference point (1210 in FIG. 12), and posY represents the y-coordinate when the upper left of the drawing area is the reference point. .. Further, width represents the width of the drawing area (1209 in FIG. 12), and height represents the height of the drawing area. In this embodiment, the object parameters are the minimum necessary for the sake of simplicity, but it is clear that other parameters (translation amount, magnification, etc.) can be added and used at the time of drawing or rendering. is there.

In S510, the CPU 100 displays the BASE64 data as an image in the drawing area 1206 based on the generated object parameters. Specifically, the CPU 100 transmits the BASE64 data corresponding to the selected stamp to the interpreter 214 of the OS layer. Then, the interpreter 214 interprets the script of the BASE64 data and displays it as a stamp image in the drawing area.

In this embodiment, for the sake of simplicity, the case where one stamp is selected is given as an example, but it goes without saying that a plurality of stamps can be selected. Further, in this embodiment, the image prepared in advance for the stamp is used, but a method of generating a drawing object in the script layer by using the Context object may be used.

<Details of stamp identification>
S24 starts when the user taps on the display 104 shown in FIG. "Tap" refers to an operation of pressing the screen with a finger on a portable information terminal, which is an operation equivalent to "clicking" by a pointing device on a PC. Here, the details of specifying the stamp of S24 in FIG. 3 will be described with reference to FIG. 7. Note that S602 to S603 are processes executed by the CPU 100 using the program of the script layer 217, and S601 is a process executed by the CPU 100 using the program of the native layer 218.

In S601, the CPU 100 acquires the coordinates tapped on the display 104 and transmits them to the script layer 217.

In S602, the CPU 100 determines whether or not the stamp added in S23 of FIG. 3 is touched from the coordinates received from the native layer 218 and the object parameter information generated in S509 of FIG. In the added stamp, the object parameter remains the default value. Therefore, according to the above example of the object parameter, the stamp is drawn in an area of 100 in the x direction and 100 in the y direction from the reference point 1210 of the drawing area 1206 as (0,0). Become. From this, among the received coordinate coordinates (x, y), the value obtained by subtracting the x-coordinate of the drawing area 1206 from the x-coordinate is in the range of 0 to 100, and the y-coordinate of the drawing area 1206 is calculated from the y-coordinate. If the subtracted value is in the range of 0 to 100, it can be determined that the stamp has been touched. When there are a plurality of stamps, the stamps displayed in the upper layer are determined in order, and the process may be terminated when the stamps are specified. When it is determined that the stamp has been touched, the stamp is in the operation acceptance state of S25 in FIG.

In the following description, it is assumed that the stamp added in S23 of FIG. 3 is touched in S24 of FIG.

In S603, the CPU 100 sets the stamp to the operation acceptance state according to the determination result. Here, setting the operation acceptance state means a state in which when an instruction for a stamp operation is given, the stamp performs an operation corresponding to the instruction. If there is no stamp in the operation acceptance state, no change occurs even if the instruction regarding the stamp operation is given. In this embodiment, only the slide bar 1204 that rotates the stamp is mentioned, but it goes without saying that other stamp operation configurations may be used.

<Details of stamp operation>
When the stamp is in the operation acceptance state, S25 in FIG. 3 starts. Hereinafter, the details of the stamp operation in S25 of FIG. 3 will be described with reference to FIG. Each step in FIG. 8 is a process executed by the CPU 100 using the program of the script layer 217.

In S701, the CPU 100 checks whether the stamp has been "swipe" based on the information that can be obtained from the touch event. Here, if it is determined that the "swipe" has been performed, the process proceeds to S702, and if not, the process proceeds to S704. Here, "swipe" refers to an operation of tapping a portable information terminal and then moving a finger without releasing it from the screen, which corresponds to a "drag" operation using a PC's pointing device. To do.

In S702, the CPU 100 updates the values of posX and posY of the object parameters of the stamp to the x-coordinate and the y-coordinate obtained by "swipe". In S703, the CPU 100 uses the object parameters to move the stamp 1209 in the drawing area 1206 to the updated coordinate position (posX, posY) based on the updated posX, posY values.

In S704, after the stamp operation, whether or not the touch of the screen is maintained is acquired from the touch event. Here, when it is determined that the touch is released, the stamp operation ends, and the process of S25 in FIG. 3 ends. On the other hand, when it is determined that the user continues to touch, the process returns to S701 and the above-described process is repeated.

In this embodiment, the stamp operation is limited to movement by a swipe event, but the present invention is not limited thereto. For example, there are many possible combinations of touch events and content operations, such as enlarging the stamp by pinching out, and translating the Canvas drawing object by combining long tap (long press) and swipe.

Here, the pinch-out is an operation in which the user touches the screen with two fingers and then the two fingers are spaced apart from each other. In a general portable information terminal, this operation is often used for enlarging an image / screen. Further, the above-mentioned movement means that the stamp 1209, which is a drawing object, moves in the Canvas, unlike the movement in the drawing area 1206. For example, in FIG. 12, if the heart in the stamp 1209 is moved to the right by width / 2, nothing is displayed on the left half of the stamp 1209, and the left half of the heart is contained in the right half. Become in a state.

<Details of printer settings>
When the user presses the print button 1205 shown in FIG. 12, the process of S26 in FIG. 3 starts. Here, the details of the printer setting of S26 of FIG. 3 will be described with reference to FIG. Note that S901 and S909 to S911 are processes executed by the CPU 100 using the program of the script layer 217, and S902, S904 to S905, S907, S908 and S912 are processes executed by the CPU 100 using the program of the native layer 218. .. Further, S903 and S906 are processes executed by the printer 112.

In S901, the CPU 100 requests the script layer 217 to the native layer 218 to acquire printer information as device information. This corresponds to a request from the script layer 217 for communicating with the printer 112. As for the request method, the native API is called from the script by the binding function as in the case of image selection. The native layer 218 is provided with a so-called wrapper that can be called directly from the script layer 217 or indirectly calls the function. For example, a native function called GetPrinterInfo is prepared, and the function is called from the script side. In this way, the native layer gets the request for communication with the external device from the script layer.

Normally, the script layer 217 cannot directly communicate with an external device due to security restrictions. Therefore, as described above, the script layer 217 once requests the native layer 218 to acquire the external device information, and communicates with the external device via the native layer 218. Here, the native layer 218 has a function of communicating with an external device (for example, a printer 112) via the OS layer 219.

In S902, the CPU 100 performs printer detection, so-called discovery, when the native layer 218 calls the function. For example, it detects printers connected by the same wireless LAN router. Here, in order to detect a printer that can communicate, for example, a protocol such as Bonjour (registered trademark) is used to attempt communication with the printer using a method such as broadcasting or multicast.

On the other hand, in S903, the printer 112 responds to the request.

In S904, the CPU 100 detects and stores the IP address of the printer that responded. Further, in S905, the CPU 100 requests the IP address of the printer that responded to provide the printer information. When there are a plurality of responsive printers, the CPU 100 requests all printers to provide information. Therefore, the CPU 100 generates a command for acquiring printer information in the native layer. The command is a command for specifying the operation of the printer, and as an example thereof, it is expressed in XML as shown below.

----------------------------------------------
01: <? xml version = "1.0" encoding = "utf-8"?>
02: <cmd xmlns: trans = "http://www.xxxx/yyyyy/">
03: <contents>
04: <operation> GetInformation </ operation>
05: </ contents>
06: </ cmd>
----------------------------------------------
Numerical values such as "01:" written on the left side of each of the above lines are line numbers added for explanation and are not described in the original XML format text.

The first line is a header, which indicates that it is described in XML format.

The cmd on the second line means the start of the command. The namespace is specified by xmlns, and the definition of command interpretation is specified. The end of the command is indicated by </ cmd> on the 6th line.

The third line is a declaration that describes the contents thereafter, and the fifth line indicates the end.

The required instruction is described on the fourth line, and the actual instruction wording exists between <operation> and </ operation>. The command wording, Get Information, is a command for acquiring information on a printer that is an external device. For example, it is requested to provide capability information such as the paper type, size, borderless printing function, print quality, etc. supported by the printer.

To generate the printer information acquisition command, for example, a fixed text stored in advance in the ROM 101 may be read. Further, it is not limited to the text format such as XML, and may be described in a binary format and communicated by a protocol according to the description. Further, the generated printer information acquisition command is transmitted to the printer 112 via the external device communication unit 221 in a format according to a communication protocol supported by the printer such as HTTP.

Moreover, the communication method is not limited to this. Connection using Wi-Fi (registered trademark) direct, Bluetooth (registered trademark), infrared communication, telephone line, wired LAN, or USB may be used, and the same effect can be obtained by communicating using a protocol according to the method. it can.

In FIG. 9, the native layer 218 is configured to generate commands, but the script layer 217 is configured to generate commands, and the same effect can be obtained. In that case, the script layer 217 creates, for example, a command including the above XML format statement and passes it to the native layer 218. In response, the native layer 218 transmits the command to the printer 112 in a format according to the communication protocol.

In S906, when the printer 112 receives a command from the information processing device 115, the printer 112 transmits the printer information, which is the device information, to the information processing device 115 in the XML format according to the communication protocol. An example of printer information is shown below.

----------------------------------------------
01: <? xml version = "1.0" encoding = "utf-8"?>
02: <cmd xmlns: trans = "http://www.xxxx/yyyyy/">
03: <contents>
04: <device id = ”Printer001” />
05: <memory receive = 7680000 />
06: <mode = 1>
07: <media> Glossy Paper </ media>
08: <size> A4 </ size>
09: <quality> 1 </ quality>
10: <border> no </ border>
11: <dpi x = 1200 y = 1200 />
12: </ mode>
13: <mode = 2>
~ Omitted ~
</ Mode>
<Mode = 3>
~ Omitted ~
</ Mode>

~ Omitted ~

</ Contents>
</ cmd>
----------------------------------------------
The first line is a header, which indicates that it is described in XML format. The cmd on the second line means the start of the command. The namespace is specified by xmlns, and the definition of command interpretation is specified. The bottom line </ cmd> indicates the end of the command.

The third line is a declaration that describes the content after that, and the content continues until </ contents> below. The device ID is shown on the 4th line. Here, it means that the model name of the printer 112 is "Printer001". The fifth line is not used in this embodiment. It will be described in detail in a later embodiment. The sixth and subsequent lines are descriptions of each mode of the printer 112. Information in one mode is described from <mode> to </ mode>. In the sixth line, the mode number is 1. Subsequent <media> describes the type of printing paper, <size> describes the paper size, <quality> describes the print quality, and <border> describes information with / without borders. The <dpi> on the 11th line represents the input resolution, which is 1200 [dpi] in the horizontal direction and 1200 [dpi] in the vertical direction. Details of the input resolution will be described later.

From the 13th line onward, information about mode 2, which is another mode, is described. As described above, the model name of the printer 112 and all the modes supported by the printer are described in this XML. The method of describing the printer information is not limited to this, and may be a text other than the tag format or another format such as a binary format.

Further, although the above describes an example in which information on the printing function of the printer is passed, the present invention is not limited to this. For example, information such as image processing and analysis processing that can be processed by the printer, presence / absence of a quiet printing mode, presence / absence of use of a memory card, and status such as the remaining amount of ink may be passed. Examples of image processing include color conversion such as monochrome, sepia, and saturation enhancement, layout of multiple images, white balance correction, noise removal, and other processing that automatically corrects a photograph to a preferable color and brightness.

Then, in S907, the CPU 100 acquires printer information from the printer. In the native layer, the CPU 100 acquires, for example, the type, size, print quality, bordered / non-rimmed items and the number of items of the printing paper in all modes of the printer 112 from the received printer information.

In S908, the CPU 100 converts the received printer information into a format that can be interpreted by the script layer 217 and transmits it to the script layer 217. That is, the CPU 100 passes the information obtained by the communication with the printer 112 to the script layer 217. Specifically, a native function is provided and the binding function is used. Further, a method such as transmitting with the received XML format printer information or changing to a text format without a tag may be used. In addition, each time a specific native function is called from the script layer 217, information may be acquired as the return value. In addition, you may pass an argument such as the mode to be acquired to the native function and obtain information as the return value. Further, the above-mentioned JSON character string may be used for delivery, or the data conversion units 207 and 203 may be used for delivery using a character string such as BASE64.

In S909, the CPU 100 forms and displays a setting screen (FIG. 13) including functions available in the printer 112 based on the printer information received from the native layer 218. This is also called display control. Here, when there are a plurality of connectable printers, the printer name is displayed, and a display screen for allowing the user to select a printer to print before displaying the setting screen 1301 is generated. Then, the CPU 100 displays the setting screen of the selected printer by using the printer information corresponding to the selected printer. The selection of the printer is not limited to the above, and a method such as selecting the printer that responded earliest, the printer having more functions, or the printer that the print job is not congested can be considered.

In this way, the CPU 100 displays the setting screen 1301 (FIG. 13) for selecting functions that can be used in the printer, such as the type / size of printing paper, print quality, and with / without borders. As an example of the method of forming the setting screen, a sample of HTML description is shown below.

------------------------------------------------
<! DOCTYPE html>
<Head>
<Title> Print settings </ title>
<Script>
<!-Paper size->
var PaperSizeNum = GetPaperSizeNum ();
var p = document.getElementById ("PaperList");
var i;
for (i = 0; i <PaperSizeNum; i ++) {
p.options [i] = new Option (GetPaperSizeT (i), GetPaperSizeV (i));
}

<!-Paper type->
var MediaTypeNum = GetMediaTypeNum ();
var m = document.getElementById ("MediaList");
var j;
for (j = 0; j <MediaTypeNum; j ++) {
m.options [i] = new Option (GetMediaTypeT (j), GetMediaTypeV (j));
}

<!-Print quality->
var QualityNum = GetQualityNum ();
var q = document.getElementById ("QualityList");
var k;
for (k = 0; k <QualityNum; k ++) {
q.options [i] = new Option (GetQualityT (k), GetQualityV (k));
}

<!-With / without rim->
var BorderNum = GetBorderNum ();
var b = document.getElementById ("BorderList");
var l;
for (l = 0; l <BorderNum; l ++) {
b.options [i] = new Option (GetBorderT (l), GetBorderV (l));
}

<!-Print function->
function printer () {
SetPrint (document.getElementById ("PaperList"). value,
document.getElementById ("MediaList"). value,
document.getElementById ("QualityList"). value,
document.getElementById ("BorderList"). value);
}
</ Script>
</ Head>

<!-Display->
<Body>
Paper size <select id = "PaperList"></select><br/>
Paper type <select id = "MediaList"></select><br/>
Print quality <select id = "QualityList"></select><br/>
With / without border <select id = "BorderList"></select><br/>
<br />

<Button id = "btn1" onclick = "printer ()"> Setting completed </ button>
</ Body>
</ html>
------------------------------------------------
The above-mentioned GetPaperSizeNum (), GetMediaTypeNum (), GetQualityNum (), and GetBorderNum () are native functions and have a function of acquiring the number of each item. For example, when the printer supports four types of paper sizes, A4, A5, B5, and L size, GetPaperSizeNum () returns 4.

GetPaperSizeT (n), GetMediaTypeT (n), GetQualityT (n), and GetBorderT (n) are also native functions, and return the character string of the value th of the argument n. For example, the return value of GetPaperSize (0), which is a function that returns paper-sized text, is "A4", and the return value of GetPaperSize (1) is "A5". These values are retrieved by the native function from the printer information received from the printer.

GetPaperSizeV (n), GetMediaTypeV (n), GetQualityV (n), and GetBorderV (n) are also native functions and return the character string of the value th of the argument n. For example, the return value of GetMediaTypeT (n), a function that returns the text of the paper type, is a wording that is displayed and shown to the user, such as "glossy paper". On the other hand, the return value of GetMediaTypeV (0) is an expression that can be interpreted by the printer as "GlossyPaper".

These words and expressions are determined by the native in association with the information sent from the printer. For example, if the value extracted from the information sent from the printer is "GlossyPaper", the text to be displayed is determined to be "glossy paper". As a method of determination, the native may hold these correspondence tables in advance and determine the text according to the correspondence table.

In the above, as an example, the specifications are such that the paper size, the paper type, the print quality, and the presence / absence of the border are set, but the specification is not limited to this. Needless to say, other setting items such as double-sided / single-sided, color / monochrome, and image correction on / off can be mentioned as other examples. In addition to the printing function as described above, information such as image processing and analysis processing that can be processed by the printer, the presence / absence of a quiet printing mode, the presence / absence of a memory card, and the status such as the remaining ink level is displayed. Is also good.

In S910, the CPU 100 selects a function to be set in the printer based on a user operation on the setting screen 1301. An example in which the HTML shown in the above example is displayed on the display 104 using the rendering unit 216 is 1301 shown in FIG. The printer information is requested via the native layer 218, and the setting screen 1301 is formed based on the information acquired from the printer information by using the above native function. The HTML may be formed by either the script layer 217 or the native layer 218.

Further, each setting item such as the paper size shown in FIG. 13 is a pull-down menu, and the item can be selected by a user operation. Here, the setting screen 1301 shows a state in which a list of items that can be selected as paper size setting items is displayed by a pull-down menu, and a paper size such as A4 or A5 can be selected by user operation. it can.

When the CPU 100 detects the user operation for the setting completion button 1302 in S911, the CPU 100 creates setting information including the setting items selected by the user operation and transmits the setting information to the native layer 218. SetPrint () in the above HTML description example is also a native function having a binding function. In the above example, SetPrint () is used to pass the settings of paper size, paper type, print quality, and with / without border to the native layer 218 as a character string.

In S912, the CPU 100 receives the setting information from the script layer 217 by the binding function. The native layer 218 later generates a print command according to the communication protocol of the printer 112 based on the received setting information, the image data to be printed, and the image data of the stamp. Then, the printer command is transmitted to the printer 112 via the communication control unit 213.

<Rendering details>
When the user presses the setting completion button 1302 on the setting screen 1301 shown in FIG. 13, the rendering process of S27 in FIG. 3 starts. Here, the details of the rendering of S27 in FIG. 3 will be described with reference to FIG.

The rendering of this embodiment is to convert the content displayed on the UI into an image size (number of pixels) for printing. The content is an image in which the above-mentioned photographic image and stamp are arranged. Normally, the number of pixels of an image used for printing is larger than the number of pixels of an image displayed on the UI, so printing requires enlargement processing of the image. Here, the enlargement processing is performed by drawing the content displayed on the UI so as to have a larger image (number of pixels) with the same composition. In this embodiment, the band rendering (division / enlargement) process is mainly executed in the script layer.

In FIG. 10, S1001, S1003, S1004, and S1007 are processes executed by the CPU 100 using the program of the script layer 217. S1002, S1005, and S1008 are processes executed by the CPU 100 using the program of the OS layer 219. S1006 is a process executed by the CPU 100 using the programs of the script layer 217 and the native layer 218.

In S1001, the CPU 100 requests the OS layer 219 to start the indicator. In S1002, the CPU 100 displays the indicator activated by the request on the display 104. This allows the user to know that the process is in progress.

In S1003, the CPU 100 determines the output size corresponding to the paper size set in the setting information determined in S911. The output size is the number of pixels of the image, and in this embodiment, the output size is calculated from the input resolution and the paper size in the print mode specified in the above settings. The input resolution is defined for each print mode of the printer 112, and image processing and quantization processing are performed based on the number of pixels of the input resolution. When image data other than the input resolution is transferred to the printer 112, the printer performs enlargement / reduction (magnification) so as to have the input resolution.

Here, an example of a method of calculating the output size will be described later assuming that the input resolution is 1200 dpi and the paper size is A4. The "number of input pixels [pixel]" corresponds to the output size, and the input resolution may differ depending on the print mode set in the printer. The print mode is a setting provided according to the paper type and quality. As another example, the output size may be predetermined for each paper size, or may be calculated from a predetermined resolution. For example, the input resolution may be set to 300 dpi or the like and calculated using the paper size.

When transferring image data equal to or lower than the input resolution in the print mode of the printer to the printer for printing, the enlargement process is executed in the printer to the input resolution in the print mode. However, when the enlargement process is executed using a known simple logic, blurring and jaggies occur in the lines of objects such as stamps in the printed image and in the photographic image, and the image quality seems to be deteriorated to the human eye. It ends up. On the contrary, even if the image data larger than the input resolution of the print mode of the printer is transferred, the image data is reduced to the input resolution in the printer.

Therefore, determining the output size from the input resolution and the paper size in the print mode of the printer as described above improves the processing efficiency and the image quality of the resulting image.

In S1004, the size of the drawing area of the output image is calculated.

Here, the drawing area of the output image is one Canvas, and the individual Canvass described above are rendered so that the size and the relative position in the area match with respect to the Canvas of the output size.

In this embodiment, the size of the drawing area of the output image is calculated according to the situation of the information processing device in which the application is operating. This is to avoid running out of memory in the information processing device. For example, when printing on a paper size of A4 with a printer having an input resolution of 1200 dpi, a capacity of 1.25 [GB] is required in the memory. The calculation method of the memory capacity is shown below.

------------------------------
Size A4: length 210 [mm] x width 297 [mm]
Number of vertical pixels [pixel]
= 210 [mm] ÷ 25.4 [mm / inch] × 1200 [dpi]
Number of horizontal pixels [pixel]
= 297 [mm] ÷ 25.4 [mm / inch] × 1200 [dpi]
Number of input pixels [pixel]
= Number of vertical pixels [pixel] x number of horizontal pixels [pixel]
Required memory capacity [Byte]
= Number of input pixels [pixel] x 3 [channel] x 8 [bits]
≒ 417.6 [MB] (for binary data)
Here, 3 [channel means that each pixel of the image data is represented by three components of RGB. If each pixel of the image data is represented by the four components of YMCK, it will be 4 [channel]. Further, 8 [bit] indicates that each color component of each pixel is represented by 8 bits having a value of 0 to 255.

Here, assuming that all 8-bit values (0 to 255) are handled by a 3-digit character string, the required memory capacity (character data) is 417.6 [MB] x 3 [characters] ≒ 1.25 [GB]. It becomes.

------------------------------
As a method of determining the size of the drawing area of the output image, for example, there is a method of trying to allocate memory, reducing the amount to be allocated if it cannot be allocated, trying again, and continuing until it can be allocated. The memory allocation is performed by setting the width (x direction) and the height (y direction) with the Cambas setAttribute and allocating the area with getContext. Here, the width is fixed to the width of the entire image output to the printer, and the above-mentioned memory allocation attempt is made while reducing the height (height). By doing so, it is possible to divide the entire image to be output into bands as illustrated in FIG.

In this embodiment, the width is fixed and the width is divided into strips in the height direction, but the present invention is not limited thereto. For example, it may be divided into a square or a horizontal direction. However, printing with a printer is generally performed in order from the top to the bottom of the image. Therefore, it is desirable that the image data to be input to the printer is also horizontally divided into strips corresponding to this printing order. In this way, each time the division / enlargement processing described later progresses, the image data can be divided and transmitted to the printer, and it is possible to shorten the time until the end of printing. From the above processing, performing rendering by dividing into bands (bands) (enlarging and drawing image data according to the input resolution of the printer) is called band rendering.

The method for determining the size of the drawing area of the output image is not limited to the above. For example, it may be set in advance to a value that is expected not to cause a memory shortage, and there is no delay in printing in rendering. You may decide on the capacity.

In this embodiment, the size of the drawing area of the output image is determined as described above to prevent the occurrence of memory shortage in the script layer.

In S1005, the CPU 100 secures a drawing area of the calculated output image in the RAM 102.

In S1006, the CPU 100 performs band rendering. The actual rendering process is executed by the interpreter of the OS layer 219 in response to the instruction from the script layer, but here, the fact that the script uses the functions of the OS is also expressed as the execution in the script layer. At that time, the photographic image is drawn in the drawing area of the output image secured in S1005. In S1006, the CPU 100 interprets the script for requesting the drawing of the photographic image and renders the photographic image. The drawing of the image is performed in the OS layer 219, and the drawImage method of the Context object of Cambas is used. Note that the drawing referred to here means to output as image data to the memory space secured as the drawing area of the output image, and does not mean to display the image on the display 104. In this embodiment, the band rendering method is used in S1006 to divide and draw the image. The details will be described below.

FIG. 14 is a diagram schematically showing the relationship between the image output to the drawing area and the band rendering.

In FIG. 14, 1401 is an image of Canvas displayed on the UI, 1402 and 1404 are stamps arranged according to the user's instruction, and 1403 is a photographic image selected by the user. The photographic image used for enlargement is not the one displayed on the UI, but the one obtained by opening the image file and reading the original image is used. As mentioned above, the display for the UI may be reduced. This is because the reduced image loses the information of the original image, and even if it is enlarged, the lost information cannot be restored. Therefore, as the image to be enlarged, an image of the image size read from the file is used instead of the image for UI. When image processing such as correction is performed as described above, the image that has been subjected to the image processing on the image read by opening the image file is targeted for enlargement. Here, the image acquired in S304 of FIG. 4 and expanded in RGB in S305 is received from the native layer 218, and the script layer 217 enlarges the image.

Some images such as stamps are stored as vector data. Since the vector data is data created as an image at the time of drawing, for example, when drawing after enlarging with Cambas, the image quality does not deteriorate. However, the stamp of the image data that is once drawn as bitmap data or is not vector data is the same as the photographic image. In this case, as in the case of the above photographic image, a source having as high a resolution as possible is used instead of being displayed in the UI.

In this embodiment, the Canvas image 1401 is enlarged to the input resolution (number of pixels) required by the printer 112. The image 1401 of Canvas before enlargement has the same size as the image 1405, and has a width of Wb [pixel] and a height (vertical) of Hb [pixel]. 1406 is an enlarged image, and has a width We [pixel] and a height (vertical) He [pixel]. Further, 1407, 1410, and 1413 are drawing areas of output images of the same size, and are width We [pixel] and height (vertical) Hp [pixel].

As described above, since the number of pixels of the enlarged image is large, memory shortage is likely to occur. Therefore, in the example shown in FIG. 14, the image data after being divided into three parts is retained or transmitted to an external device (for example, a printer). That is, here, an example will be described in which a memory that can be stored if the capacity is 1/3 of the capacity of the image data after the enlargement processing can be secured as the drawing area of the output image. In this embodiment, the enlarged image is divided into three for a brief explanation, but the present invention is not limited thereto, and in reality, the drawing area of the output image depends on the state of the memory of the execution environment. The number of divisions should be determined.

Next, the band rendering process as an example of division and enlargement will be described with reference to the flowchart.

FIG. 15 is a diagram showing details of the band rendering process. In FIG. 15, S1501 to S1506 and S1508 are processes in which the CPU 100 executes the program of the script layer 217, and S1507 is a process in which the CPU 100 executes the program of the native layer 218.

First, in S1501, the number of divisions is calculated. The number of divisions is determined according to the following formula. That is,
Number of divisions = He [pixel] / Hp [pixel]
Is. However, the numbers after the decimal point are rounded up. Here, Hp is the height of the output image area calculated in S1004.

In S1502, the enlargement ratio is determined.

The enlargement ratio is the ratio of the image 1406 to the size of the Canvas displayed on the UI (images 1401 and 1405), and is as follows. That is,
ScaleW (horizontal magnification) = We / Wb
ScaleH (vertical magnification) = He / Hb
Is.

In S1503, "0" is assigned to the variable i, and loop processing is executed for the number of divisions.

Next, in S1504, the movement amounts of the stamps 1402 and 1404 and the photographic image 1403 included in the image 1401 are set. The movement is an upward parallel movement, and the movement distance is Hbp × i. Here, since i = 0 at the first time, no movement is performed. A detailed description of the movement will be described later.

In S1505, the enlargement ratio is set. The enlargement ratio in this embodiment is set as follows by using the scale () method of Cambas.

scale (ScaleW, ScaleH);
Then, in S1506, the drawing area of the output image is output.

In this embodiment, the image data is output to the drawing area of the output image by the drawImage () method of Cambas. At that time, the entire image 1406 does not fit in the drawing area 1407 of the output image. Therefore, the drawImage () outputs the image data from the upper end of the image 1406 to the drawing area of the output image, and ends the output when the lower end of the drawing area of the output image is reached. Therefore, in the drawing area of the output image, an image in which the upper part is cut out from the dotted line portion of the image 1406 is output to the drawing area 1407.

In S1507, the image data in the drawing area of the output image is output to a file. In this case, the script layer 217 outputs to a file via the native layer 218. This output form is an RGB bitmap format in this embodiment. Since the storage location of the file is the secondary storage device 103, there is no direct influence on the memory (RAM 102) in terms of capacity. Therefore, there is no problem with insufficient memory (RAM).

Here, the binding function calls the native layer function from the script layer. At this time, the number of vertical and horizontal pixels of the image data to be saved is also passed as an argument. The native layer saves the image data for the number of passed vertical and horizontal pixels in a file. As mentioned above, since the script layer holds the numerical value as a character string, it is converted to binary data in the native layer and then saved in a file.

In the above example, the image data is stored as a file in the secondary storage device 103, but the present invention is not limited thereto. For example, it may be saved in a memory card, an external storage device, a server, or a cloud system via the Internet. In addition, due to OS restrictions, the script layer 217 cannot secure a large amount of memory, but the native layer 218 may be able to handle a large amount of memory. In that case, the divided and enlarged image data may be held in the memory (RAM) managed by the native layer 218.

In S1508, it is checked whether or not the processing for the number of divisions is completed. Here, if it is determined that all the processing for the number of divisions has not been completed, the value of i is incremented by "+1" and the processing returns to S1504. If all the processing is completed, the band rendering processing is terminated.

Next, the expansion of the output center portion in the three divisions will be described. This corresponds to the second loop processing (i = 1).

In S1504, the movement amounts of the stamps 1402 and 1404 and the photographic image 1403 included in the image 1401 are set. The movement is an upward parallel movement, and the movement distance is Hbp × i. Here, the Translate () method of Canvas is used for movement. Here, Hbp is the height when the image 1401 is divided into three parts. It can be expressed by the following formula. That is,
Hbp = Hp × Hb / He
Is.

The virtual diagram when the parallel movement is set in this way is 1408 in FIG. The thick line frame is the range output by Cambas, and the outside of the thick line frame is not output (or drawn). Here, as shown in 1408 of FIG. 14, the image 1401 is set to be translated upward by Hbp.

Then, similarly to the enlargement of the upper part described above, in S1505, the enlargement ratio is set by the scale () method of Canvas. The magnification of enlargement is the same as the above-mentioned upper part. 1409 in FIG. 14 is a virtual diagram when enlarged at the enlargement ratio. By sliding upward as shown in 1408 in FIG. 14 and setting the enlargement with the scale () method, the enlarged image is translated upward by Hp. The settings were made in the order of movement and enlargement, but the order of enlargement and movement may be used. The amount of movement at that time is a value (Hp in this example) that matches the enlarged image.

Subsequently, in S1506, the image data is output to the drawing area of the output image by the drawImage () method of Canvas in the same manner as described above. As described above with reference to 1406 to 1407 of FIG. 14, only the portion above the dotted line portion of the thick line frame of 1409 of FIG. 14 is output to the drawing area of the output image. The result of the output is shown as 1410 in FIG. 14, which is the central portion of the three divisions.

Similar to the upper part, the output result is sent to the native layer, and in S1707, it is output to a file.

Finally, the expansion of the lower part of the output in three divisions will be described. This corresponds to the third loop processing (i = 2).

In S1504, as in the case of the central portion described above, the translate () method is used to set the original Canvas image 1401 to be translated upward. The distance of parallel movement is Hbp × 2. The reason why the movement amount is Hbp × 2 is that the image corresponding to the height of Hbp of the original image is enlarged in the output at the upper part, and the image corresponding to the height of Hbp of the original image is enlarged in the output at the center. This is because it has already been output. 1411 of FIG. 14 is a virtual representation of this parallel movement.

Subsequently, in S1505, the enlargement setting is performed using the scale () method as in the central portion described above. 1412 of FIG. 14 shows the state of the expansion virtually.

Subsequently, in S1506, the image data is output to the drawing area of the output image by the drawImage () method of Canvas, as in the central portion described above. Similar to the above, only the part above the dotted line portion of the thick line frame of 1412 in FIG. 14 is output to the drawing area of the output image. The result of the output is 1413 in FIG. 14, which is the lower part of the three divisions. Output this to a file with S1507 in the same way as the upper part and the central part.

Here, an example of just dividing into three is shown, but other cases can also be dealt with. For example, the divided bottom image may be less than the height (Hp) of the drawing area of the output image, that is, it may not be divided and a remainder may appear. At that time, parallel movement, enlargement, and output are performed in the same manner as described above. At this time, there is no image in the lower part of the drawing area of the output image in which the lowermost part is output. When this output result is saved in a file in the native layer 218, the area where this image does not exist is omitted and saved.

In S1007, the CPU 100 requests the OS layer 219 to stop the indicator. In S1008, the CPU 100 stops the indicator and erases it from the display 104.

<Print details>
In S28 of FIG. 3, the print process is executed using the image data after rendering. Here, the details of the print of S28 of FIG. 3 will be described with reference to FIG. Note that S1101 to S1103 of FIG. 11 are processes executed by the CPU 100 using the program of the native layer 218, and S1104 is a process executed on the printer side.

In S1101, the CPU 100 divides and enlarges the image by the band rendering of S1006 in FIG. 10, and converts the RGB image of each band saved as a file into a format that can be used by the printer 112. The formats that can be used by the printer include RGB, JPEG, and CMYK, as well as the original formats of printer vendors such as PDF. Any form of them may be used here.

In S1102, the CPU 100 generates a command to be transmitted to the printer 112 based on the setting information and the image data converted in S1101. The command includes an image format, the number of pixels of each image (We, Hp in FIG. 14), and image data. The first command sent to the printer 112 includes the number of pixels of the entire image (We, He in FIG. 14). In S1103, the CPU 100 sequentially transmits commands of a plurality of bands generated in S1102 to the printer 112 selected by the user according to the communication protocol that can be used by the printer. By the above processing, output processing using commands of a plurality of bands (a plurality of image data) is realized.

The printer 112 stores the image sent as a command in the memory in the printer 112. The printer 112 waits until all the images are prepared. When all the images are aligned, the printer 112 combines the images into a single image for output. After generating one image data using the RGB image file of each band, the image data may be converted into a format that can be used by the printer. However, only when the memory that can perform the processing in the native layer is available.

In S1104, the printer 112 prints the combined image.

In the above example, the printer 112 waits for the printing operation until all the images are aligned, but the printer 112 may print each time it receives the divided images. By receiving the next image data sent as a command while printing, the time until the end of printing can be shortened. Further, in the above example, the divided and enlarged image is temporarily saved as a file, but it may be sequentially transmitted to an external device (printer or the like) without saving. By doing so, it becomes possible to shorten the time until the end of printing and to print without accessing the secondary storage device.

Therefore, according to the embodiment described above, even in an environment in which a memory shortage occurs in the script layer in the hybrid application and the image data cannot be enlarged, the image data can be enlarged by the image data division process. This makes it possible to print a high-quality image.

[Embodiment 2]
Here, an example in which division enlargement (band rendering) is performed entirely in the native layer will be described. This embodiment realizes the step of FIG. 3 as in the above-described first embodiment, and there are many parts where the processing overlaps with that of the first embodiment. Therefore, for the sake of brevity, the description of the overlapping portion with the first embodiment will be omitted, and only the portion characteristic of this embodiment will be described here.

The only characteristic part of this embodiment is the rendering of S27 in FIG. In the first embodiment, most of the rendering is processed by the script layer 217 and the OS layer 219 requested by the script layer 217, but in this embodiment, the rendering is handled by the native layer 218, and high-speed processing is realized. ..

FIG. 16 is a flowchart showing a rendering process according to the second embodiment. In FIG. 16, the same steps as described in FIG. 10 in the first embodiment are designated by the same step reference numbers, and the description thereof will be omitted. In FIG. 16, S1001, S1003, and S1004A are processes in which the CPU 100 executes the program of the script layer 217, and S1002 and S1008 are processes executed by the CPU 100 in the OS layer 219. Further, S1005A, S1006A, and S1006B are processes in which the CPU 100 executes the program of the native layer 218.

After S1001 to S1003, in S1004, the CPU 100 requests the native layer 218 for rendering. At this time, the object that holds the image in vector data such as a stamp is enlarged by the script layer 217 and drawn. Image data such as the drawn stamp is passed to the native layer 218 in the BASE64 format, and these are converted into binary data in the native layer 218. At this time, the drawing of the stamps is executed one by one and passed to the native layer 218, which converts this into binary data, and then moves on to the drawing of the next stamp. In this way, only one image can be retained as character string data.

Holding image data as character string data as described above requires a large amount of memory capacity as compared with binary data. Therefore, the memory capacity used can be reduced by the above processing. The native layer 218 may temporarily store the received image data as a file in the secondary storage device, and open the file when using the file. By doing so, the amount of memory used temporarily can be reduced.

Further, here, the position information of each object (photograph image, stamp, etc.) is sent from the script layer 217 to the native layer 218. The position information is the relative position and size of each object, and the order of front and back, which is similar to the arrangement displayed in the UI. The position information is interpreted by the native layer 218 to determine the expanded position and size of each object.

In S1005A, the capacity of the drawing area of the output image is calculated. This is basically the same process as S1004 in FIG. 10, but the following restrictions are provided here.

As described in the first embodiment, the printer information received from the printer 112, which is an external device, has the description <memory receive = 7680000 /> on the fifth line of the above-mentioned XML. This indicates that the maximum amount of data that the printer 112 can receive at one time is 7680000 [Byte] (= 7,68 [MB]). Therefore, the capacity of the drawing area of the output image needs to be less than or equal to this value. In reality, since headers and footers are added before and after the image data and transmitted to the printer 112, the capacity of the image area is reduced accordingly. Not limited to the maximum amount of data that can be received, the size of the image that can be expanded and held by the printer in RGB may be acquired from the printer, and the capacity of the drawing area described later may be determined.

In S1006A, a drawing area of the output image is secured. As the drawing area, only the capacity calculated by S1005A is secured in the memory area managed by the native layer 218. Band rendering is executed in S1006B. The details will be described later.

Subsequent processing is as described in the first embodiment with reference to FIG.

FIG. 17 is a flowchart showing details of the band rendering process according to the second embodiment. This band rendering is performed in the native layer 218, and as can be seen by comparing FIG. 17 with FIG. 15, the same processing performed in the script layer 217 in the first embodiment is executed in the native layer 218 in the second embodiment. There is. Therefore, here, only the part characteristic of the second embodiment will be described.

In S1701, the number of divisions is calculated in the same manner as in S1501. The number of divisions is determined by the size of the drawing area of the output image. Since the size is determined based on the value contained in the printer information received from the printer 112 which is an external device, the number of divisions is also based on the value received (or specified / instructed) from the external device. It can be said that it has been decided.

In S1702, the enlargement ratio and each image position are determined. Here, the rate of converting the size of the photographic image opened from the file to the size of the input to the external device is calculated. The above-mentioned position information is used for the calculation. Although it is expressed as an enlargement ratio, the magnification may be less than 1 and the reduction may occur. As for the position, the position after the division and expansion is obtained from the variable i and the above position information. The arrangement of image data of stamps other than the photographic image is also calculated from the position information. Since the stamp image is enlarged in the script layer 217, it is not necessary to enlarge it in the native layer 218.

Similar to S1503, S1703 starts loop processing with the number of divisions.

In S1704, the image data is expanded and arranged in the drawing area of the output image while being enlarged based on the determined enlargement ratio and position information, and the images are combined according to the order of the front and back surfaces. At this time, the stamps are arranged and combined in the same manner, but there is no need to enlarge them.

In S1705, the divided and enlarged image output to the drawing area of the output image is transmitted to the external device (printer 112) at any time. At this time, the size of the image data is added to the header of the transmission data. The transmission follows the protocol supported by the external device via the external device communication unit 221. When the printer 112 receives the divided image, it prints at any time. When transmitting the next divided image, the native layer 218 inquires the printer 112 whether or not the next divided image can be received. In response to the inquiry, the printer 112 notifies the native layer whether or not the next divided image can be received. As described above, the divided image is transmitted, received, and printed at any time.

In S1706, it is checked whether or not all the band renderings for the number of divisions are completed. Here, if it is not finished, the process returns to S1704, and if all is finished, the band rendering process is finished.

Therefore, according to the above-described embodiment, the amount of data that can be received by the external device is received, the drawing area size of the output image is calculated in consideration of the amount, and this is secured in the memory. It is possible to execute the enlargement processing of the image data while preventing the occurrence. In addition, it is possible to prevent memory overflow in an external device.

Further, according to this embodiment, since the enlargement processing is executed in the native layer, the image enlargement processing can be executed at a higher speed than the enlargement processing of the photographic image in the script layer, and the processing time can be shortened.

In this application, the command for the printer is transmitted in the native layer, but it may be transmitted in the OS layer.

[Another Embodiment]
Further, the example of the information processing device shown in FIG. 1 is hardware assuming a portable mobile computer (smartphone or tablet terminal), but the present invention is not limited thereto. For example, it is possible to realize the same configuration on hardware such as a stationary personal computer, a server, a game machine, and a digital camera.

In addition, although the example of the printer as an external device has been described in the above embodiment, the present invention is not limited thereto. For example, as an external device, other smartphones, tablet terminals, PCs, servers, game machines, and the like are also targeted.

Furthermore, in the above embodiment, as the drawing of the content (photographic image or stamp image), the drawing of the content is described by giving an example by the Cambas function of Javascript, but the drawing of the content is not limited to this. For example, it is also possible to draw the content by using SVG (Scalable Vector Graphics).

Further, in the embodiment described above, an example in which one image is selected from the image folder in the device has been described, but the present invention is not limited to this. For example, an image may be selected by specifying an absolute path of data, specifying each folder containing an image, or taking an image on the spot using the camera function of the device. Further, as the data acquisition destination, selection of an image on the Internet, selection of an image in a removable storage medium, acquisition of an image by communication with an external device, and the like can be mentioned.

In addition, as the printer of the above embodiment, an inkjet printer, a laser printer, a sublimation printer, a dot impact printer and the like can be used. These may be so-called multifunction printers, which have a printer function, a scanner function, and the like and are not a single function.

The functions of the above embodiments can also be realized by the following configurations. That is, it is also achieved by supplying the program code for performing the processing of the present embodiment to the system or device, and executing the program code by the computer (or CPU or MPU) of the system or device. In this case, the program code itself read from the storage medium realizes the function of the above-described embodiment, and the storage medium storing the program code also realizes the function of the present embodiment.

Further, the program code for realizing the function of the present embodiment may be executed by one computer (CPU, MPU), or may be executed by a plurality of computers collaborating with each other. May be good. Further, the program code may be executed by a computer, or hardware such as a circuit for realizing the function of the program code may be provided. Alternatively, a part of the program code may be implemented by hardware and the rest may be executed by a computer.

201: Image acquisition unit, 202: Image reading unit, 203: Data conversion unit

Claims (31)

A hybrid application program including a first program layer in which an instruction set to be translated and executed by a processor is described, and a second program layer in which an instruction set pre-translated other than the processor is described. , An information processing device executed by the processor.
A division / enlargement means that executes division processing and enlargement processing on image data,
It has an output means for executing an output process for an external device using a plurality of image data obtained by the division process and the enlargement process by the division / enlargement means.
The instruction set of the first program layer includes any of HTML, CSS, and Javascript.
The division / enlargement means of the first program layer sets position information of image data and size information of image data using a Web drawing language, and divides the image data based on the position information and the size information. An information processing device characterized by performing processing and enlargement processing. The first aspect of claim 1, wherein the output means generates one magnified image data based on the plurality of image data, and transmits print data based on the one magnified image data to the external device. Information processing device. The information processing device according to claim 1, wherein the output means sequentially transmits the plurality of image data to the external device. Further have a display means Ru is displayed on the display unit the drawing result of the image data,
The display means, the information processing apparatus according to any one of claims 1 to 3, wherein Rukoto included in the first program layer. The information processing apparatus according to any one of claims 1 to 4, wherein the plurality of image data obtained by the division / enlargement means are stored in a hard disk or a flash memory. The information processing apparatus according to any one of claims 1 to 5, wherein the additional image added to the image data is enlarged by the first program layer. The division / enlargement means
A first determining means for determining the magnification of an image based on the resolution of the printer as the external device, and
It is characterized by including a second determining means for determining the number of divisions of the image based on the available capacity of the memory included in the printer and the capacity that can be secured in the storage means of the information processing apparatus. The information processing device according to any one of claims 1 to 6. The first program layer treats image data as text data and treats it as text data.
The information processing apparatus according to any one of claims 1 to 7, wherein the second program layer handles the image data as binary data. The information processing apparatus according to any one of claims 1 to 8, wherein the drawing process for the image data uses Cambas or Scalable Vector Graphics (SVG). A receiving means for receiving an image processing instruction for the image data from the user,
Further have a image processing means for performing image processing according to an instruction of image processing is accepted by the accepting means,
The receiving means is included in the first program layer, and is included in the first program layer.
Wherein the image processing means, the information processing apparatus according to any one of claims 1 to 9, wherein Rukoto included in the second program layer. The information processing apparatus according to claim 10, wherein an image represented by vector data is added to the original image by image processing by the image processing means. The information processing device according to any one of claims 1 to 11, wherein the Web drawing language includes either HTML or SVG. Further having a display control means for displaying an indicator based on a user's instruction,
A claim characterized in that, after the image data for the specified number of divisions obtained by the division processing and the enlargement processing by the division / enlargement means is generated, the display control means erases the indicator from the display screen. Item 2. The information processing apparatus according to any one of Items 1 to 12. The first specifying means for specifying the first size of the drawing area and
A second specifying means for specifying a second size that is the size of the enlarged image and is larger than the first size, and
Further, it has a determination means for determining the number of divisions based on the first size and the second size.
The information according to any one of claims 1 to 13, wherein the division process and the enlargement process are executed by the division / enlargement means until the image data of the number of divisions determined by the determination means is obtained. Processing equipment. The division / enlargement means is characterized in that the division processing and the enlargement processing are executed by moving the image data based on the position information of the image data and then performing the enlargement processing based on the size information. The information processing apparatus according to claim 14. A hybrid application program including a first program layer in which an instruction set to be translated and executed by a processor is described, and a second program layer in which an instruction set pre-translated other than the processor is described. , A processing method of an information processing device executed by the processor.
In the first program layer, the image data is divided and enlarged , and the image data is divided and enlarged.
Output processing to an external device is executed using the plurality of image data obtained by the division processing and the enlargement processing.
The instruction set of the first program layer includes any of HTML, CSS, and Javascript.
The feature is that the position information of the image data and the size information of the image data are set by using the Web drawing language, and the division process and the enlargement process are executed on the image data based on the position information and the size information. Processing method to do. A computer that executes a hybrid program including a first program layer written in a web standard language and a second program layer written in a programming language different from the web standard language is divided into image data. It functions as an output means for executing an output process to an external device using the division / enlargement means for executing the enlargement processing and the plurality of image data obtained by the division processing and the enlargement processing by the division / enlargement means.
The instruction set of the first program layer includes any of HTML, CSS, and Javascript.
The division / enlargement means of the first program layer sets position information of image data and size information of image data using a Web drawing language, and the image data is described based on the position information and the size information. A computer-readable program comprising performing a division process and an enlargement process. 17. The output means according to claim 17, wherein the output means generates one magnified image data based on the plurality of image data, and transmits the print data based on the one magnified image data to the external device. program. The program according to claim 17, wherein the output means sequentially transmits the plurality of image data to the external device. Further it makes the computer function as display means Ru is displayed on the display unit the drawing result of the image data,
The program according to any one of claims 17 to 19 , wherein the display means is included in the first program layer . The program according to any one of claims 17 to 20, wherein the plurality of image data obtained by the division / enlargement means is stored in a hard disk or a flash memory. The program according to any one of claims 17 to 21, wherein the additional image added to the image data is enlarged by the first program layer. A first determining means for determining the magnification of an image based on the resolution of the printer as the external device, and
A feature of further functioning the computer as a second determining means for determining the number of divisions of the image based on the available capacity of the memory included in the printer and the capacity that can be secured by the storage means of the computer. The program according to any one of claims 17 to 22. The first program layer treats image data as text data and treats it as text data.
The program according to any one of claims 17 to 23, wherein the second program layer treats the image data as binary data. The program according to any one of claims 17 to 24, wherein the drawing process for the image data uses Canvas or Scalable Vector Graphics (SVG). A receiving means for receiving an image processing instruction for the image data from the user,
The computer is further made to function as an image processing means that performs image processing according to the image processing instruction received by the receiving means .
The receiving means is included in the first program layer, and is included in the first program layer.
Wherein the image processing means, the program according to any one of claims 17 to 25, characterized in Rukoto included in the second program layer. The program according to claim 26, wherein an image represented by vector data is added to the original image by image processing by the image processing means. The program according to any one of claims 17 to 27, wherein the Web drawing language includes either HTML or SVG. The computer is further made to function as a display control means for displaying an indicator based on a user's instruction.
After the image data for the specified number of divisions obtained by the division processing and the enlargement processing by the division / enlargement means is generated, the display control means causes the indicator to be erased from the display screen. Item 10. The program according to any one of Items 17 to 28. The computer further
The first specifying means for specifying the first size of the drawing area and
A second specifying means for specifying a second size that is the size of the enlarged image and is larger than the first size, and
It functions as a determining means for determining the number of divisions based on the first size and the second size.
The program according to any one of claims 17 to 29, wherein the division process and the enlargement process are executed by the division / enlargement means until the image data of the number of divisions determined by the determination means is obtained. .. The division / enlargement means is characterized in that the division processing and the enlargement processing are executed by moving the image data based on the position information of the image data and then performing the enlargement processing based on the size information. 30. The program according to claim 30.