JP6868185B2 - Image processing equipment and programs - Google Patents

Description

The present invention relates to an image processing apparatus and a program.

Multifunction devices equipped with functions such as copy, printer, and fax are becoming more multifunctional, and it is required to efficiently construct an entire system including various applications (applications).

Patent Document 1 describes an image processing device that allows a user to easily grasp whether or not an application with an icon displayed on an application list screen is being executed. For the icon for calling the application and the macro that registered the operation settings of the application, the screen information storage means for storing the list screen information including the icon position, the icon image, the application identification information, and the macro identification information, and the job A screen control means that acquires the status information of the application whose status has changed from standby to execution, outputs identification information, and makes a screen update request, and makes a screen update request to the list screen of the application created based on the list screen information. When received, a screen creation means for updating the display form of the application icon indicated by the acquired identification information to a display form different from the icons of other applications, and a display means for displaying a list screen including the updated icon. It is stated that

Patent Document 2 describes that the operability of a user is improved in a multifunction device having multiple functions. Arrangement information storage means for storing icon arrangement information in which the standard application, the extended application, and the application identification information that identifies the standard and extended applications are associated with the coordinate information of each icon and the image information of the icon, and the icon arrangement information. A list screen creation means for creating an application list screen in which icons corresponding to standard and extended applications are displayed, a display means for displaying the created application list creation screen, and a standard corresponding to the pressed icon. Alternatively, it is described that the extended application is specified from the icon arrangement information, and the specified standard or extended application is provided with screen control means for requesting the display of the operation screen.

Further, for launching a plurality of applications, for example, the following techniques are known.

In Patent Document 3, a plurality of independent programs are operated in cooperation with each other, and when the sub-process is started, the start information of the sub-process is created and the return destination information of the main process is set. It is described that the memory of is released and the sub-processing is started.

Patent Document 4 provides a file transfer program with a function of invoking another program in application linkage processing in file transfer, starts another application before and after file transfer, transfers the processing result, and processes the transfer result. It is stated that it should be done.

Japanese Unexamined Patent Publication No. 2012-248102 Japanese Unexamined Patent Publication No. 2012-27662 Japanese Unexamined Patent Publication No. 05-073326 Japanese Unexamined Patent Publication No. 05-216737

Considering the versatility or expandability of the image processing device, in addition to the applications built into the system from the beginning (internal applications), existing applications outside the system and applications provided by third parties (external applications) are appropriately added. Where it is required to be able to be added, it is required that the screen control of these external applications can be performed in the same manner as the internal application. In particular, for internal applications, it is possible to improve the visibility and operability of users by displaying multiple screens according to a predetermined logical hierarchical structure and giving visual effects when transitioning screens. However, it may be required to control the screen of an external application while maintaining such a visual effect.

An object of the present invention is to provide an image processing device and a program capable of screen control even when displaying a background color when different types of applications such as an internal application and an external application coexist.

The invention according to claim 1 comprises a first application and a second application operating on the framework, the first application and the second application, and a control means for executing the framework, and the control means. Displays the first window of the first application together with the background color by executing the framework, and the background color when the second window of the second application is displayed in the hierarchy below the first window. Is set to transparent, and the control means is an image processing device that sets the background color to be transparent when a fault display or a banner is displayed in the first window of the first application.

The invention according to claim 2 is the image processing apparatus according to claim 1, wherein the control means sets the background color to be non-transparent when the second window of the second application is not displayed.

The invention according to claim 3 includes a first application and a second application operating on the framework, the first application and the second application, and a control means for executing the framework, and the control means. Displays the first window of the first application together with the background color by executing the framework, and the background color when the second window of the second application is displayed in the hierarchy below the first window. Is set to transparent, and the control means embeds a dummy management element for managing the window of the second application in the first window of the first application in response to an instruction to start the second application. This is an image processing device that displays the second window of the second application and transparently displays the dummy management element.

The invention according to claim 4 is the image processing apparatus according to claim 3 , wherein the control means executes the second application with an operation on the transparently displayed dummy management element as an operation on the second application. is there.

The invention according to claim 5 is the image processing apparatus according to any one of claims 1 to 4 , wherein the first application is a pre-embedded application, and the second application is an addable application.

According to the sixth aspect of the present invention, the first application is separated into a core logic unit responsible for basic processing and a UI frame unit responsible for drawing processing, and the core logic unit is an interface (I / I /) defined by the framework. The image processing apparatus according to claim 5 , which implements F).

The invention according to claim 7 is a program that causes a processor that controls an image processing device to execute the processor, the first application and the second application that operate on the framework, and a control means that executes the framework. To function as
The step of displaying the first window of the first application together with the background color by executing the framework, and the background color when the second window of the second application is displayed in the hierarchy below the first window. Is a program that executes a step of setting the background color to be transparent and a step of setting the background color to be transparent when a fault display or a banner is displayed in the first window of the first application.

According to the inventions of claims 1, 2 and 7 , the screen can be controlled even when the background color is displayed when different types of applications such as the first application and the second application coexist. In addition, the screen can be controlled in the case of fault display or banner display.

According to the invention of claim 3 , the first application and the second application can be seamlessly screen-controlled.

According to the invention of claim 4 , further, the user's operation on the second application can be surely executed.

According to the inventions of claims 5 and 6, the pre-embedded application and the addable application can be screen-controlled.

It is a functional block diagram of an image processing apparatus. It is a logical configuration diagram of the system. It is explanatory drawing which shows an example of a home screen. It is a logical configuration diagram of an application. It is a logical block diagram of a conventional system. It is a block diagram of the application on the framework. It is explanatory drawing which shows the specific configuration example of an application. It is explanatory drawing which shows the specific configuration example of the application list. It is explanatory drawing which shows the pattern of a core logic and a UI frame. It is explanatory drawing at the time of UI and logic change. It is explanatory drawing which shows the pattern of the application on the framework. It is a system configuration diagram of life cycle management including a booter and a starter. It is a flowchart of life cycle management. It is a graph which shows the time from the system startup until the home screen is displayed. It is a logical configuration diagram when an external application (external application) is implemented. It is another logical configuration diagram when an external application is implemented. It is a block diagram of a companion application. It is a manifest information explanatory diagram of a companion application. It is a block diagram of the framework. It is a block diagram of window management information. It is a start sequence diagram of an external application. It is an extrusion sequence diagram of an external application. It is a redisplay sequence diagram of an external application. It is a sequence diagram at the time of normal termination of an external application. It is a sequence diagram at the time of abnormal termination of an external application. It is a sequence diagram at the end of an external application from the outside. It is a hierarchical structure explanatory diagram when the background color of the internal browser is black. It is a superimposition display state explanatory diagram of an internal browser and an external application when the background color is black. It is an explanatory view (fault) of the superimposed display state of the internal browser and the external application when the background color is transparent. It is a superimposition display state explanatory diagram (banner) of the internal browser and the external application when the background color is transparent. It is a sequence diagram of a fault display. It is a sequence diagram of banner display. It is operation event transfer explanatory diagram in banner display.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<Overall system configuration>
FIG. 1 is a block diagram of a configuration of an image processing system including an image processing device according to the present embodiment. The image processing system includes a terminal device 10 and an image processing device 12. The terminal device 10 and the image processing device 12 are connected via the communication means 14. As the communication means 14, a data communication network such as a LAN (local area network) is used.

The terminal device 10 is connected to the image processing device 12 via the communication means 14, and transmits a print job or the like including a document printing command according to a user's instruction.

The image processing device 12 includes a ROM 16, a RAM 18, an HDD 20, a control unit 22 composed of one or a plurality of CPUs, an input / output interface I / F 24, an operation unit 26 such as a touch panel, and an image processing unit 28.

The control unit 22 composed of one or a plurality of CPUs receives a print job instruction or the like from the terminal device 10 via the input / output interface I / F 24 according to the processing program stored in the ROM 16, interprets the PDL data, and intermediates the data. Data is generated, and drawing data (raster data) is further generated from the generated intermediate data. Further, the control unit 22 executes various commands such as copy, scan, and fax received from the operation unit 26.

The image processing unit 28 includes a print module, a scanner module, a fax module, a paper paper feed module, a document paper feed module, and an image processing accelerator.

The print module is a module having a function of outputting an image on paper. For example, it has a known inkjet method configuration and prints drawing data on paper. Liquid or molten solid ink is ejected from a nozzle or the like, and recording is performed on paper, film, or the like. There are two methods for ejecting ink: the drop-on-demand method (pressure pulse method), which ejects ink using electrostatic attraction, and the pressure generated by forming and growing bubbles due to high heat. There is a thermal inkjet method and the like. The recording head includes, for example, a head that ejects cyan ink, a head that ejects magenta ink, a head that ejects yellow ink, and a head that ejects black ink, and each head has a width at least equal to the width of the paper. The head is used. Ink droplets of each color are ejected onto an intermediate transfer body by a recording head for recording, and then transferred to paper for printing.

A scanner module is a module that reads an image from paper and converts it into electronic data.

A fax module is a module that includes a modem and an image processing module for fax, and executes a fax function.

The paper paper feed module is a module that transports paper from a paper tray to a print module.

The document paper feed module is a module that transports paper from the document tray to the scanner module.

The image processing accelerator is a module that performs compression / decompression processing in conjunction with a scanner module or the like. This image processing accelerator is not essential and may be an additional module.

In addition to these, the image processing device 12 is composed of a finisher for punching and sorting paper, a USB, an IC card reader, and the like, and is composed of an authentication unit, a billing unit, a motion sensor, and a face camera that authenticate users. Etc. may be provided.

Further, the image processing device 12 may be connected to the Internet via the communication means 14, and may be provided with Ethernet (registered trademark) or Wi-Fi (registered trademark).

<Logical structure of the program>
FIG. 2 shows a logical configuration of a system executed by the control unit 22. The system is roughly divided into two layers, a presentation layer 30 and a device service layer 32.

The presentation layer 30 is a layer for implementing various applications, and includes a framework 31 and various applications. Framework 31 is a group of execution environment software that enables a JavaScript (registered trademark) application to run on a computer system. More specifically, JavaScript is executed on a Web browser, and Base Frame and UI Frame are loaded as HTML iframes. The application is JavaScript software that implements the I / F provided by the framework 31. Framework 31 manages the life cycle of various applications. That is, the framework 31 creates a base frame for various applications, reads the core logic of the various applications, and initializes (initializes) the core logic. Instruct. In addition, when the system is shut down, the core logic of various applications is instructed to finalize, and the base frame is discarded. The core logic and life cycle management of various applications will be described later.

The device service layer 32 is a layer that manages various hardware devices, and the various hardware devices include the print module of the image processing unit 28 and the like.

FIG. 3 shows an example of a screen (home screen) displayed on the operation unit 26 of the image processing device 12. On the home screen, copy (Copy) button 34, ID card copy (ID Copy) button 36, scan (Scan) button 38, fax (Fax) button 40, my copy (MyCopy) button 42, web application (Web App1) The icons of the button 44 and the easy copy button 46 are displayed. When the user touches any button to select it, the application assigned to each button is started and the screen of the application is displayed. The user can recognize that each application corresponds to each button.

As described above, the application is JavaScript software that provides the I / F defined by the framework 31, and is a component that directly provides the function to the user. It has a common configuration defined by framework 31. In addition, each application is configured so that the degree of coupling with other applications is low. Some applications work with the user through the user interface (UI), while others do not. An application that collaborates with a user proactively executes display and input through the presentation layer 30.

The figure also shows a login button 48 for the user to log in, and the application also supports this button.

<Application implementation structure>
FIG. 4 shows the structure of the application. Application 50 is roughly divided into three components. That is, it is separated into a core logic (Core Logic) 52, a UI frame (UI Frame) 54, and a manifest file (Manifest file) 56. Here, "separation" does not mean that they are physically separated, but that they are logically separated.

Core Logic 52 is a component that performs basic processing (basic behavior and cooperation between applications) as an application, and is always present in each application. Core Logic provides the I / F defined by Framework 31.

The UI Frame 54 is a component for drawing and displaying as an application, and is specifically managed as a display window.

Manifest file 56 is a list of static information for each application. The static information is the application identifier (ID), display name, icon image, version, creation date, and the like. The manifest file 56 includes a manifest file 56-1 for core logic and a manifest file 56-2 for UI frames. One of the information described in the manifest file 56 is the isLaunchable attribute. This attribute determines whether or not it is displayed as an icon (button) on the home screen.
Display with isLaunchable = true
It is hidden when isLaunchable = false.

In such a configuration, the rules of communication between the Core Logic 52 and the UI Frame 54 are as follows.
(1) The core logic (Core Logic) 52 communicates with another core logic (Core Logic) 52.
(2) The UI Frame 54 communicates only with the Core Logic 52.

Therefore, the UI Frame 54 does not communicate with other UI Frames 54.

FIG. 5 shows a conventional program structure. Conventionally, a huge root application (RootApp) 60 is prepared to provide various functions used by each application, and all applications depend on this root application 60. In addition, a device application (Device App) 62 that specializes in various device states also exists separately, and almost all applications depend on this device application 62. Furthermore, implementations are becoming more common among applications, and there are also dependencies between applications. Therefore, even when an application is added or deleted, adjustment is required between the applications each time, and the root application 60 is always required to be modified, so that the application cannot be easily added or deleted.

On the other hand, FIG. 6 shows the program structure of the present embodiment. Each application is separated into a Core Logic 52, a UI Frame 54, and a Manifest file 56, and the Core Logic 52 of each application is connected to the framework 31. The UI Frame 54 of each application is connected to the Core Logic 52 of the application.

For example, in the case of a copy application, the copy application is separated into a core logic (Core Logic) 52, a UI frame (UI Frame) 54, and a manifest file (Manifest file) 56, and the core logic (Core Logic) (Core) is taken as an example. The Logic 52 is connected to the framework 31, and the UI Frame 54 is connected to the Core Logic 52. The coupling between each application is limited and there is no dependency as in the past, and the cooperation between each application is executed by the framework 31 via Core Logic 52. Since the Core Logic 52 of each application provides the I / F defined by the framework 31, even when a new application is added, the I / F defined by the framework 31 should be provided. Can be easily added with. In addition, since the coupling between applications is limited, it is easy to delete the application.

FIG. 7 shows an example of a copy application. In the figure, baseframe.html is the core logic (Core Logic) 52, and base_manifest.json is the manifest file 56-1 of the core logic (Core Logic) 52. Further, uiframe.html is the UI frame (UI Frame) 54, and app_manifest.json is the manifest file 56-2 of the UI frame (UI Frame) 54.

FIG. 8 shows an example of an application list. In the figure, "base" indicates the manifest file 56-1 of the core logic 52, and "app" indicates the manifest file 56-2 of the UI frame 54. The "type" in the manifest file 56-2 indicates the type of application. The types of applications are as follows.
That is, the application
STD: Application installed as standard OT: Shortcut of application installed as standard (STD) EXT: Application that can be added (Part 1)
CS: Applications that can be added (Part 2)
There are four types. The application installed as standard is an application corresponding to copy (Copy), scan (Scan), fax (Fax), etc. shown in FIG. Further, a special companion application is assigned to each of the OT, EXT, and CS applications, and each companion application has its own function. Each companion application also has a Core Logic 52 like the STD application. By including the application type in the manifest file 56, the internal implementation of each application can be distinguished from each other.

Further, as described above, "isLaunchable" in the manifest file 56-2 is attribute information for determining whether or not to display the icon on the home screen. In the figure,
isLaunchable = true
This means that a copy button is displayed.

Since the application is separated into a core logic 52 and a UI frame 54, it can be said that the application list describes the correspondence between the two.

Since the manifest file 56 is created for each application, it is desirable to set an identifier indicating the type for each application and a unique identifier within the type. For example, in the copy application manifest file,
type: STD
ID: copy
And so on. type is an identifier indicating the type (standard application), and ID is a unique identifier.

Further, the manifest file 56 also includes information required at startup and information necessary for drawing the home screen as static information. The information required at startup is the storage destination information of the core logic (Core Logic) 52 and the storage destination information of the UI frame (UI frame) 54, and the framework 31 is the storage destination information of the core logic (Core Logic) 52. Load Core Logic 52 with reference to. Further, the Core Logic 52 loads the UI frame 54 as necessary by referring to the storage destination information of the UI frame 54.

The information required for drawing the home screen is the icon button storage destination information and the button display order.

The manifest file 56 is referred to by the application management component of the device service layer as described later, and is used for creating the application list.

FIG. 9 shows a pattern of the implementation structure of the application.

FIG. 9A is a pattern in which the core logic 52 exists but the UI frame 54 does not exist. Companion applications, etc. are applicable, not standard applications. FIG. 9D is an application list corresponding to FIG. 9A.

FIG. 9B is a pattern in which the core logic 52 and the UI frame 54 exist, and each is a pattern having a one-to-one correspondence. FIG. 9 (e) is an application list corresponding to FIG. 9 (b).

On the other hand, in FIG. 9C, although there are a core logic 52 and a UI frame 54, a plurality of UI frames 54 have a common core logic 52. This is the case. The UI Frame 54 determines the display form when displaying a button on the home screen, but even when displaying a plurality of buttons, the implementation is efficient by sharing the Core Logic 52. Will be implemented. In addition, maintainability is improved by sharing the Core Logic 52 in a plurality of applications. There is no limit to the number of UI Frames 54 that share the Core Logic 52. FIG. 9 (f) is a list of applications corresponding to FIG. 9 (c). Specific examples of the manifest file 56-1 include, for example,
{
"id": "appId.std.copy",
"url": "app / copy / baseframe / baseframe.html"
}
A specific example of the manifest file 56-2 is, for example,
{
"subId": "copy",
"type": "STD",
"appUrl": "app / copy / copy / uiframe.html",
"isLaunchable": true,
"orderWeight": 100,
"largeIcon": "common / img / preset / app / app_copy_120.png",
"smallIcon": "common / img / preset / app / app_copy_48.png",
"displayName": "Copy",
"displayNameId": "001"
}
Or
{
"subId": "idcopy",
"type": "STD",
"appUrl": "app / copy / idcopy / uiframe.html",
"isLaunchable": true,
"orderWeight": 900,
"largeIcon": "common / img / preset / app / app_idcardcopy_120.png",
"smallIcon": "common / img / preset / app / app_idcardcopy_48.png",
"displayName": "IDCardCopy",
"displayNameId": "002"
}
Is.

In addition, in FIG. 9B and FIG. 9C, whether or not the button is actually displayed on the home screen by setting the isLaunchable attribute value of the manifest file 56-2 of the UI Frame 54. Is decided. For example, in FIG. 9C, there are a first UI Frame 54 and a second UI Frame 54 having a common Core Logic 52, and the first UI If isLaunchable = true in the manifest file for UI Frame 54, while isLaunchable = false in the manifest file for the second UI Frame 54, the former is displayed as a button, but the latter Do not show.

Since the core logic 52 and the UI frame 54 are separated as the execution structure of the application, only the UI frame 54 is changed without changing the core logic 52. However, it is possible to easily customize the display form on the screen of the application.

FIG. 10 shows an example in which the display form on the screen is customized.

FIG. 10A is an initial display form. Focusing on the ID card copy application, its UI Frame 54 is idcopy / uiframe.html, and its manifest file 56-2 is idcopy / app_manifest.json.

FIG. 10B shows a case where the display form is customized. In the ID copy application, the UI frame 54 and the manifest file 56-2 are replaced with idcopy_for_xxx / uiframe.html and idcopy_for_xxx / app_manifest.json for the new display form. Of course, it is also possible to replace only the manifest file 56-2.

On the other hand, FIG. 10C shows a case where the logic of the application is changed instead of the display form. In this case, the core logic 52, the UI frame 54, and the manifest file 56 are all replaced with new ones. That is, replace copy and below with copy_for_xxx.

FIG. 11 shows a pattern of a specific application implementation structure including the framework 31.

FIG. 11A is an example of a pattern when the copy application and the ID copy application are implemented. The copy application is separated into a core logic 52 and a UI frame 54, and the core logic 52 communicates with the framework 31. The UI frame 54 communicates only with the Core Logic 52. Similarly, the ID copy is also separated into a core logic (Core Logic) 52 and a UI frame (UI frame) 54, and the core logic (Core Logic) 52 communicates with the framework 31. The UI frame 54 communicates only with the Core Logic 52.

FIG. 11B shows another example in which a print application is implemented in addition to the copy application and the ID copy application. The copy application and the ID copy application are separated into a common Core Logic 52 and each UI frame 54. That is, the copy application and the ID copy application communicate with the framework 31 via the common Core Logic 52. Further, in the print application, although the core logic 52 exists, the UI frame 54 does not exist. FIG. 11 includes all the patterns shown in FIG.

In the conventional application implementation structure, the core logic 52 and the UI frame 54 are not separated in this way, and processing and screen drawing are mixed and difficult to understand. In addition, there is no common I / F for applications, and each application publishes the I / F without permission and refers to it without permission. On the other hand, in the embodiment, the framework 31 defines the application I / F, and the core logic 52 of each application always implements the application I / F. Therefore, the I / F is different from the conventional one. It can be said that the direction of F is different. Further, not only the communication between the framework 31 and the application but also the communication I / F between the applications can be realized by the I / F disclosure function and the I / F reference function provided by the framework 31.

In theory, there may be a pattern in which the UI frame 54 is shared by a plurality of applications and the core logic 52 is provided individually. However, in this case, since the structure becomes complicated when viewed from the framework 31, it is not particularly mentioned in the embodiment. Of course, this does not necessarily mean the exclusion of the pattern.

<Application life cycle management>
FIG. 12 shows a basic configuration for managing the life cycle of each application by the framework 31. Here, the framework 31 is an application execution environment.

In the presentation layer, the framework 31 and various applications 50 exist, and the booter 60 and the starter application 64 exist. In addition, the application management component 62 exists in the device service layer.

The Booter 60 is a component that manages the start / end of the entire presentation layer. The framework 31 is initialized and started by the Booter 60.

The application management component 62 provides the framework 31 with a list of applications based on the manifest files 56 of the various applications 50.

The starter application 64 is an application that implements the starter I / F70 defined by the framework 31. This starter application 64 is the only application that exists in the system, and is called from the framework 31 when the initialization of all the applications 50 is completed.

As described above, the various applications 50 are copy applications, ID copy applications, fax applications, and the like, and include Core Logic 52. The Core Logic 52 of the various applications 50 implements the application I / F 72 defined by the framework 31.

Specifically, the application I / F implemented by each application 50 is
-Initialize
・ Finalize
-Window Pushed Out
-Window Prepare Exposed
-Processing when deleting a window (windowPrepareTerminated)
Is. Each application 50 implements a handler for these events.

The framework 31 is a Java Script component (referred to as a communication control component) for enabling method publishing / calling and event publishing / subscription / publishing among the core logic 52 of various applications 50. ) Is provided. A method can be defined to take any argument and return an arbitrary return value. Published methods are managed independently for each application. The application on the method caller can confirm the completion of method processing by calling back. In addition, events can be defined by each application with arbitrary data. Published events are managed independently for each application. More specifically, the communication control component allows Core Logic 52 to publish and call methods, define and issue events, and register listeners, expose methods with "on", and methods with "off". Stop publishing. The exposed method can be called by call. For example, the first application "on" an I / F with a certain application and publishes it to the framework 31, and the second application targets the published I / F of the first application with respect to the framework 31. And "call".

A more specific specification (API) for publishing / calling a method and publishing / subscribing / issuing an event is as follows.

The object of Arena com which is a Java Script component is arenaCom, and the method is exposed by arenaCom.on (methodName, methodFunc). The argument methodName is the name of the method to be published, and methodFunc is the entity of the method processing to be published. The published method can be called by call.

Stop publishing methods with arenaCom.off (methodName). methodName is the name of the method to stop publishing.

Call the method exposed by arenaCom.call (appid, methodName, args, callbacl). Appid is the application ID of the method publishing source, methodName is the public method name to be called, args is the argument, and callback is the callback called when the method processing is completed.

Publish the event with arenaCom.publishEvent (eventName). eventName is the name of the event to be published. The event can be registered as a listener immediately after it is released. Also, make the event private by arenaCom.unpublishEvent (eventName). eventName is the name of the event to be kept private.

Register the listener for the public event with arenaCom.addListener (appid, eventName, listenerFunction, completeCallback). Appid is the ID of the application that publishes the event, eventName is the name of the event to be received, listenerFunction is the entity of the process called when the event occurs, and completeCallback is the callback called when the listener registration is completed.

Fire an event with arenaCom.fireEvent (EventName, data, completeCallback). EventName is the name of the event to fire, data is the data associated with the event, and completeCallback is the callback called when firing all listeners for the event is complete.

Fire an event for a specific listener with arenaCom.fireEventTo (listenerid, eventName, data, completeCallback). listenerid is the application ID of the event notification destination, and completeCallback is the callback called when firing the specific listener for the event is completed.

FIG. 13 shows a flowchart of life cycle management of various applications by the framework 31.

When the Booter 60 boots the framework 31, the framework 31 requests the application list from the application management component 62 of the device service layer and acquires the application list from the application management component 62.

When the framework 31 acquires the application list, it creates a base frame for each application according to this list and loads various applications 50 including the starter application 64 (load phase). That is, the core logic (Core Logic) 52 of each application is read. Specifically, the framework 31 loads the core logic 52 with reference to the storage destination information of the core logic 52 defined in the manifest file 56. The base frame is a frame for executing the core logic 52 of each application, and this frame is not displayed. The loading order of the Core Logic 52 of each application is arbitrary and random. When all applications have completed registration of the application I / F implementation, they move to the next phase.

It is assumed that the methods and events of each application itself are published before the registration process of the application I / F implementation.

Next, the framework 31 instructs each application to initialize (initialize) through the application I / F (initialization phase). Specifically, the framework 31 issues an "app" event and an "initialize" method for each application. When all the applications have called the completion call callback for the initialization instruction, the framework 31 notifies the Booter 60 of the completion of the initialization process and proceeds to the next phase. The initialization order of each application is also arbitrary. In this initialization process, each application executes data acquisition for the device service layer.

Next, the Booter 60 instructs the framework 31 to start providing a function by the application, and the framework 31 receives the instruction to start the starter application 64 (start phase). The starter application 64 acquires the information of the initial startup application managed by the device service layer and displays the initial screen. This phase completes when the starter application 64 calls the completion call callback for the start instruction.

When the system is terminated, the framework 31 instructs the core logic 52 of each application to finalize (terminate). Also, the base frame for each application is discarded.

In the load phase, the core logic 52 of each application is read in no particular order, so even if an application is added, it is not necessary to change this load phase. In addition, in the initialization phase, all applications are initialized, so it is guaranteed to call other applications, and individual meetings are not required. In this way, since there is no waiting between applications and only the core logic 52 having a relatively small size is loaded, the system startup time and the application startup time are shortened.

When each application publishes its own I / F, the initialization level differs for each application because the start, pre-initialization processing, initialization, post-initialization processing, stop, pause, etc. are different for each application. As a result, the timing at which the application can be called varies. In particular, before calling the application, it becomes necessary to confirm whether or not the other party can be called, which complicates control. On the other hand, in the embodiment, the initialization time can be shortened as described above, and the startup time of the home screen after the initialization can also be shortened.

FIG. 14 shows a comparison of the time from the system startup to the display of the home screen in the conventional and the embodiments.

In the past, the application initialization time required a waiting time in addition to the pure initialization time, and the startup time of the home screen also required a waiting time in addition to the pure startup time. In the embodiment, the pure initialization time can be shortened and the waiting time can be reduced. The same applies to the home startup time. Conventionally, when there is a dependency between applications, it is necessary to make an adjustment so that a deadlock does not occur. However, in the embodiment, since there is no such dependency, the deadlock adjustment is also unnecessary.

As described above, in the present embodiment, the application is separated into the core logic (Core Logic) 52 and the UI frame (UI frame) 54, and the I / F defined by the framework 31 is divided into the core logic (Core Logic) 52. In the configuration to be implemented, the Core Logic 52 communicates with the Core Logic 52 of another application via the framework 31, while the UI frame 54 is the core of the application. By configuring to communicate only with the logic (Core Logic) 52, each application has a common configuration defined by the framework 31, and at the same time, the degree of coupling with other applications is low. It becomes possible to add or delete easily.

Next, a case of implementing an application that can be added will be described. As mentioned above, there are applications that can be added in addition to the applications that are installed as standard. The application installed as standard is software that provides the I / F defined by the framework 31, and can be said to be an internal application. On the other hand, the application that can be added is an external application that does not necessarily provide the I / F defined by the framework 31 because it can be an existing application or an application provided by a third party. Therefore, it is possible to implement an application that can be added later (hereinafter referred to as "external application") without making major modifications, and even if the operation of the external application becomes unstable or abnormal. Even if this happens, it is desirable that it does not affect the operation of other internal applications.

Therefore, as described above, a special companion application is assigned to the external application for each type of the external application, and the companion application executes the external application by intervening between the framework 31 and the external application.

FIG. 15 shows the logical configuration of a system including an external application and a companion application. It corresponds to FIG. In addition to the home application (home application) 70 for displaying the home screen and the fault application (fault application) 72 for controlling the screen at the time of abnormality, the presentation layer 30 includes a companion application (companion) as an internal application (internal application). App) 74, 76, 78 will be implemented. Further, as an external application (external application), an external application A80, an external application B82, and an external application C84 are implemented. The companion application 74 is a companion application corresponding to the external application A80, the companion application 76 is a companion application corresponding to the external application B82, and the companion application 78 is a companion application corresponding to the external application C84. Each companion application 74, 76, 78 is software that mediates between the framework 31 and the external application A80, the external application B82, and the external application C84, and the framework 31 requests the external application A80, the external application B82, and the external application C84. It has a function to absorb the conditions to be applied. When the button on the home screen is operated by the user, the companion apps 74, 76, 78 activate the external apps A80, B82, C84 in response to the request from the home app 70. Since the companion apps 74, 76, and 78 absorb the demands as an internal application, the requirements as an internal application can be satisfied without modifying the external application A80, the external application B82, and the external application C84.

FIG. 16 shows another logical configuration. Instead of implementing a companion application for each external application, a single companion application 75 manages a plurality of external applications. The companion application 75 is a companion application corresponding to the external application A80, the external application B82, and the external application C84. The companion application 75 is software that mediates between the framework 31 and the external application A80, the external application B82, and the external application C84, and the conditions required by the framework 31 for the external application A80, the external application B82, and the external application C84 are satisfied. It has a function to absorb.

When launching an external application, it is desirable to maintain the same screen transitions and operability as the internal application from the viewpoint of seamlessly constructing the entire system. For this reason, the internal application is started in the internal browser, and the external application is started in the external browser so that each application is started in different browsers, and even if an abnormality occurs in the external browser, the internal browser is affected. The framework 31 manages not only the internal browser but also the external browser collectively, and more specifically, treats the external browser as a kind of window in addition to the normal iframe-based window and manages it uniformly. .. In this way, by managing the internal browser and the external browser collectively and controlling both as windows, the screen control information of the internal application and the external application is unified, so that the control is simplified.

In this embodiment, the framework 31 manages not only the internal browser but also the external browser collectively, and treats the external browser as a kind of window in addition to the normal iframe-based window and manages them in a unified manner. However, it is also possible to build a companion application as a system that has a supervisor-like function that switches between an internal browser and an external browser depending on the situation.

FIG. 17 shows a functional block diagram of the companion application 74. The same applies to the companion apps 75, 76, 78.

The companion application 74 includes an external application meta information management unit 741, an operating condition inspection unit 742, an external application start instruction unit 743, an external application process management unit 744, and an event detection unit 745 as functional modules.

The external application start instruction unit 743 provides the external application button information to the home application 70 via the framework 31, and when the user operates the external application button, the external application start instruction unit 743 receives the external application button from the home application 70 via the framework 31. Receive a launch command. Upon receiving the Launch command, the external application start instruction unit 743 provides the framework 31 with a window generation and window display command, and updates the information of the external application process management unit 744.

The external application meta information management unit 741 stores the manifest information of the companion application 74.

The operating condition inspection unit 742 checks the presence or absence of the authority when receiving the activation command.

The external application process management unit 744 stores the external application process generated by the window generation command.

The event detection unit 745 receives the external application end request command.

In addition to this, the companion application 74 receives a notification when the external process terminates abnormally, and activates the abnormal termination detection unit that updates the management information of the external application process management unit 744, or detects logout and the like. It may be provided with a context change detection unit or the like that terminates the external application inside and clears the cache.

FIG. 18 shows an example of manifest information stored in the external application meta information management unit 741. In the figure, "type" indicates the type of application, and "EXT" indicates that it is an external application. In addition, "subid" indicates an ID within the type. In addition, "appUrl" indicates a URL opened by an external browser. Furthermore, "toolsicon" and "displayName" indicate the icon and name to be displayed on the home screen.

FIG. 19 shows a functional block diagram of the framework 31. The framework 31 includes an event transmission / reception unit 311, a screen control unit 312, an application discrimination unit 313, a window management unit 314, a hierarchy management unit 315, and an application management unit 316 as functional modules.

The screen control unit 312 controls the entire flow. That is, the entire control of returning the result from the reception of the window generation request from the outside is performed. In addition, window management information is generated and the information is added to the window management unit 314.

The application discrimination unit 313 calls a function to be called using the type of application. That is, it is called according to the internal function or the external function. Specifically, a table that defines the correspondence between the type of application and the function is stored, and the table is used to distinguish between them.

The hierarchical management unit 315 manages the hierarchical structure. The hierarchy management unit 315 manages the active windows of each hierarchy. Specifically, the hierarchy management unit 315 manages the hierarchy structure in the order of application layer / pop-up layer / banner layer / alert (low level) layer / alert (high level) layer / fault layer, and external application of the internal application. Insert a window below the application layer.

The window management unit 314 manages the information of each window using the window ID as a key. The window information includes internal window information and external application window information.

FIG. 20 shows an example of window management information managed by the window management unit 314. The window management information is composed of common management information 314a, internal window management information 314b, or external application window management information 314c. In the case of an internal application, it is composed of common management information 314a and internal window management information 314b, and in the case of an external application, it is composed of common management information 314a and external application window management information 314c. The common management information 314a includes a window ID, a display state, and a hierarchy id that defines the hierarchy. The internal window management information 314b includes an html div element and an html iframe element of the internal window. The external application window management information 314c includes the html div element of the external window and the process ID of the device service layer (D layer). Here, the html div element of the external window is a div element for embedding the external window in the internal window. The html div element of this external window is an empty div element with only a transparent frame, so to speak, it functions as a dummy div element. Seamless screen control is possible by inserting this dummy div element into the screen hierarchy in the same way as handling the iframe of the internal application.

The operation of each functional module of the companion application 74 and each functional module of the framework 31 will be described in more detail.

FIG. 21 shows a sequence from when the button of the external application on the home screen is operated by the user until the external application is displayed. This is the processing of the user, the home application 70, the companion application 74 (or the companion application 75, 76, 78), the framework 31, and the device service layer (device layer) 32.

When the user presses the external application button, the home application 70 transmits the launch command of the button pressed by the user and its accompanying information to the companion application 74. The accompanying information includes the ID of the external application, that is, the subid of the manifest information.

The external application start instruction unit 743 of the companion application 74 receives the launch command from the home application 70, and acquires the external application meta information from the external application meta information management unit 741 based on the external application ID. Further, the external application start instruction unit 743 instructs the operation condition inspection unit 742 to check whether or not the specified start condition of the external application is satisfied. The operating condition inspection unit 742 of the companion application 74 acquires the current user from the framework 31 and the authority of the current user from the authority information management unit of the device service layer 32, and the current user is designated. Check if you have the authority to start the external application. The external application start instruction unit 743 of the companion application 74 issues a window creation command (createWindow) to the framework 31 when the current user has the authority to start the specified external application. At this time, detailed information of the external application, specifically, the type of the external application and its ID are provided.

When the screen control unit 312 of the framework 31 receives the window creation command createWindow) from the companion application 74, the application determination unit 313 determines the function to be used for window information generation. The screen control unit 312 executes the window generation function for an external application determined by the application determination unit 313 to generate window management information. The window management information is composed of common management information 314a and external application window management information 314c. The external application window management information 314c also includes a dummy div element for embedding on the internal side as described above. The screen control unit 312 adds the generated window management information to the window management unit 314. The screen control unit 312 instructs the device service layer 32 to start the external application based on the management information. Specifically, it commands the start request of the external browser together with the type and ID of the external application.

The device service layer 32 starts the designated external application and returns the process ID to the framework 31. The device service layer 32 hides the generated external application window and arranges it in a layer lower than the internal browser window.

When the framework 31 receives the process ID, it returns the window ID of the external application to the companion application 74 as a result of the window creation command createWindow).

The companion application 74 then issues an external application display request (showWindow) to the framework 31.

When the screen control unit 312 of the framework 31 receives the external application display request (showWindow), the display process of the specified window is started, and if there is another window displayed in the same hierarchy, the other window is not displayed. Start display processing (extrusion processing). If the specified window is an external application window, the div created as a dummy or alias is displayed on the internal side, and the external application window is displayed. Then, the window designated by the hierarchy management unit 315 is registered as the window of the lowest hierarchy. Further, the screen control unit 312 determines whether or not to display the internal browser window. That is, the hierarchy management unit 315 is inquired to check the windows displayed in each hierarchy, and if windows such as banners and faults are displayed on the inside other than the lowest hierarchy, the internal browser is displayed. If not, hide the internal browser (in the figure, the internal side is abbreviated as LUI).

FIG. 22 shows a sequence of extrusion processing.

An external application window is displayed on the screen, and it is assumed that the user presses the home key from this state.

When the device service layer 32 detects the pressing of the home key by the user, the device service layer 32 provides the framework 31 with the pressing of the home key.

When the event transmission / reception unit 311 of the framework 31 receives the event of pressing the home key, the event transmission / reception unit 311 notifies the home application 70 to that effect.

When the home application 70 receives the home key pressing event, the home application 70 specifies the window ID of the created home screen to the framework 31 and calls the window display (showWindow).

When the screen control unit 312 of the framework 31 receives the window display (showWindow), the display process of the specified window is started, and if there is another window displayed in the same hierarchy, the display process of the other window is hidden. To start. If the hidden window is an external application window, the div created as a dummy or alias to the inside is hidden, and the external application window is hidden. Then, the companion application 74 is notified of the push event windowPusheOut). Needless to say, this push event notification is a notification that the external application window has been hidden. As a result, the external application window is hidden by the display command of the home screen.

When the companion application 74 receives the push event (windowPushedOut), the event detection unit 745 detects this and updates the necessary processing, specifically, the internal window state management.

FIG. 23 shows a sequence diagram when an external application that has already been started exists and is hidden when the user presses the external application button on the home screen.

When the user presses the external application button on the home screen, the home application 70 notifies the companion application 74 of the launch command of the pressed button and its accompanying information, that is, the external application ID (subid in the manifest information).

Upon receiving the Launch command, the external application launch instruction unit 743 of the companion application 74 inquires whether or not the external application designated by the external application process management unit 744 has been started. When the specified external application has already been started, the external application start instruction unit 743 acquires the window ID corresponding to the external application ID from the external application process management unit 744, and requests the framework 31 to display the external application (showWindow). ) Is issued.

When the screen control unit 312 of the framework 31 receives the external application display request showWindow), the display process of the specified window is started, and if there is another window displayed in the same hierarchy, the other window is hidden. Start processing (extrusion processing). If the specified window is an external application window, the div created as an internal dummy or alias is displayed and the external application window is displayed. Then, the window designated by the hierarchy management unit 315 is registered as the display window of the lowest hierarchy. Further, the screen control unit 312 determines whether or not to display the internal browser window. That is, the hierarchy management unit 315 is inquired to check the windows displayed in each hierarchy, and if a window such as a fault or banner is displayed other than the lowest hierarchy, the internal browser is displayed, otherwise the internal browser is displayed. Hide the internal browser.

FIG. 24 is a sequence diagram when the external application is normally terminated, and is a sequence diagram when the user presses the end button in the external application.

When the user presses the end button of the external application, the external application notifies the device service layer 32 of the end request.

When the device service layer 32 receives the termination request from the external application, the device service layer 32 notifies the framework 31 of the termination request.

When the event transmission / reception unit 311 of the framework 31 receives the end request including the process ID, the event transmission / reception unit 311 instructs the window management unit 314 to convert the process ID to the window ID. Then, the event transmission / reception unit 311 issues a window end preparation event (windowPrepareTerminated) to the companion application 74.

The event detection unit 745 of the companion application 74 receives the window end preparation event from the framework 31 and prepares for the end of the external application. That is, the history and cache of the browser are cleared, and the listener is deleted from the authentication application. The event detection unit 745 notifies the framework 31 that the window is ready to end.

When the event transmission / reception unit 311 of the framework 31 receives the window end preparation completion notification from the companion application 74, it issues a process end request to the device service layer 32 and deletes the displayed window of the layer management unit 315. In addition, the window information is deleted from the window management unit 314. Further, the event transmission / reception unit 311 issues a window termination event (windowTerminated) to the companion application 74.

When the event detection unit 745 of the companion application 74 receives the window end event from the framework 31, it instructs the external application process management unit 744 to delete the management information of the process, and requests the framework 31 to display the initial screen. Issue.

When the application management unit 316 of the framework 31 receives the initial screen display request from the companion application 74, the application management unit 316 issues the initial screen display request to the home application 70 (initial start application). As a result, the home screen is displayed as the initial screen. A screen other than the home screen may be set as the initial screen.

FIG. 25 is a sequence diagram when the external application terminates abnormally, and is a sequence diagram when the process of the external application terminates abnormally due to some cause such as insufficient memory or a segmentation fault. In some cases, the user can see the external application in operation, and in other cases, it exists in the background due to the extrusion process.

When the operating system (OS) detects the abnormal termination of the external application, the device service layer 32 notifies the framework 31 of the abnormal termination of the external application.

When the event transmission / reception unit 311 of the framework 31 receives the external application abnormal termination notification including the process ID, it instructs the window management unit 314 to convert the process ID to the window ID, and the external application window is being displayed. The displayed window of the hierarchy management unit 315 is deleted. Further, the event transmission / reception unit 311 deletes the window information from the window management unit and issues a window termination event (windowTerminated) to the companion application 74.

When the event detection unit 745 of the companion application 74 receives the window end event from the framework 31, the external application process management unit 744 is instructed to delete the management information of the process, and other necessary post-processing is performed. Further, the event detection unit 745 issues an initial screen display request to the framework 31 when the external application is being displayed.

When the application management unit 316 of the framework 31 receives the initial screen display request, it issues the initial screen display request to the home application 70 (initial start application). In the case of abnormal termination of the external application, an error notification may be made with a banner or a fault instead of the home screen as the initial screen.

FIG. 26 is a sequence diagram when the application is terminated from the outside, and is a sequence diagram when the user sets the prohibition of using the external application from the Web UI of the image processing device.

The user sets to prohibit the use of external applications on the Web UI. The WebUI notifies the device service layer 32 of the external application use prohibition setting.

When the event detection unit 745 of the companion application 74 receives a notification from the device service layer 32 that the external application usage setting is prohibited, the external application process management unit 744 determines whether or not there is a running external application. Instruct to check. When the running external application exists and is being displayed, the event detection unit 745 issues an initial screen display request to the framework 31 and also issues a window termination event (terminateWindow). Its argument is the window ID.

The screen control unit 312 of the framework 31 receives the window termination event (terminateWindow), deletes the displayed window of the hierarchy management unit 315, and deletes the window information from the window management unit 314 when the external application is being displayed. To do. It also issues a process termination request to the device service layer 32. The device service layer 32 terminates the external application in response to this process termination request. Further, the application management unit 316 receives the initial screen display request from the companion application 74, and issues the initial screen display request to the home application 70 as the initial screen activation application. As a result, the external application is terminated and the home screen is displayed as the initial screen.

Here, when transitioning from one screen to another, such as when deleting a screen, the appearance is not good without the background color (background color), so the screen transition should be made by changing the background color to black, for example. Is effective.

FIG. 27A schematically shows the hierarchical structure of windows by the internal browser controlled by the screen control unit 312 and the hierarchical management unit 315 of the framework 31. As described above, the screen control unit 312 and the layer management unit 315 manage the hierarchical structure in the order of application layer / pop-up layer / banner layer / alert (low level) layer / alert (high level) layer / fault layer. , The figure shows only the app layer and the fault layer for simplification. When a layer 88 of the background color (background color) is arranged under the application layer and the window is displayed only by the internal browser, the background color is set to black, for example (hatched area in the figure). Therefore, when the screen transitions so that the application window of the application layer contracts to the center of the window, for example, the user sees black, which is the color of the background, as the color of the margin area during the transition.

FIG. 27B shows an example of a screen when a user presses a specific application button on the home screen to start a specific application. The window 90 of the home screen is reduced, and the window 92 of a specific application (copy in the figure) is displayed on the home screen 90 at the same time. At this time, the background color (black) 94 is displayed below the home screen and the application screen.

On the other hand, if black is displayed as the background color, when the window of the external application is placed under the application layer of the internal browser, the black background color hinders the user's viewpoint, and the external application under the application layer is displayed. A situation may occur that cannot be visually recognized. Of course, when the internal browser is hidden, the background color is naturally hidden, so it is possible to see the external application, but the fault display in the fault layer and banner layer above the application layer. When displaying a banner or banner, the internal browser is displayed, so the background color is also displayed, and the external application becomes invisible due to the background color.

FIG. 28 shows a hierarchical structure in which an external application is arranged in a layer below the application layer to display it, and an internal browser is also displayed in order to display a fault or a banner. Due to the presence of black as the ground color layer 88, the user cannot see the external application. In the figure, a cross indicates that it is invisible.

Therefore, when the external application is arranged and displayed in the hierarchy below the internal browser, when it is necessary to display the internal browser, the framework 31 changes the background color from normal black to another color, for example, half. Make the external application visible by changing it to transparent or transparent. When it is necessary to display the internal browser when displaying the external application, specifically, when displaying the fault in the fault layer or when displaying the banner in the banner layer.

FIG. 29A schematically shows a hierarchical structure in the case of performing fault display when the external application is arranged and displayed in the layer below the internal browser. The framework 31 sets the internal browser to the display state to perform fault display, and makes the ground color layer 88 transparent, for example.

FIG. 29B shows the screen display at this time. Since the background color is transparent, even if you display Fault 96 (for example, message display when the document feeder cover is open), you can still see external apps placed in the hierarchy below the background color. it can.

FIG. 30A schematically shows a hierarchical structure in which a banner is displayed when the external application is arranged and displayed in the layer below the internal browser. The framework 31 displays an internal browser in a display state and displays a banner, and sets the background color to, for example, transparent.

FIG. 30B shows the screen display at this time. Since the background color layer 88 is transparent, even if the banner 98 is displayed, the external application arranged in the layer below the background color can be visually recognized. In the case of the fault display shown in FIG. 29, the user cannot operate the application because it is an abnormality of the device itself, but in the case of the banner display shown in FIG. 30, the user continues because it is not an abnormality of the device itself. You may want to operate an external application. In this case, the user operation on the external application can be detected by using the dummy transparent div element embedded in the application layer of the internal browser. This point will be described later.

FIG. 31 shows a sequence diagram in the case of performing a fault display. It is a sequence diagram when the user releases the interlock of the device, for example, in the state where the external application is started and the window of the external application is displayed.

When the user releases (opens) the interlock, the device service layer 32 detects this and notifies the fault application of the fault occurrence event.

When the fault application receives the event of the fault occurrence, it requests the framework 31 to create a fault screen (createWindow). The fault screen is requested as the highest layer. In addition, the fault application requests the framework 31 to display the fault screen (showWindow).

When the screen control unit 312 of the framework 31 receives the request from the fault application, the screen control unit 312 performs normal processing as an internal application and displays a window. That is, it creates an iframe and loads the fault screen html file. Then, the iframe is displayed in response to the screen display request from the fault application. At this time, the screen control unit 312 inquires of the window management unit 314 and checks whether or not there is an internal window to be displayed and whether or not there is an external application window to be displayed. Then, the following display processing is executed according to the check result.
(A) When there is no internal window to be displayed and there is an external window to be displayed:
Hide the internal browser and display the external application window (b) If there is an internal window to be displayed and there is an external window to be displayed:
Display the internal browser, set the background color (background color) to transparent, and display the external application window (c) When there is an internal window to be displayed and there is no external application window to be displayed:
Display the internal browser and do not display the external application window (d) When there is no internal window to be displayed and there is no external application window to be displayed:
Show the internal browser, not the external app window

In the figure, the screen control unit 312 sets the background color to be transparent, assuming that the case (b) above corresponds to the case. By making the background color transparent, even if the internal browser is displayed, the external application window in the lower hierarchy can be displayed so that the user can see it.

In the case of (c) above, the reason why the background color is not made transparent and remains black even when the internal browser is displayed is that the background color needs to be made transparent because there is no external application window to be displayed. Because there is no. Needless to say, even when the state (b) is changed to the state (c), the background color is changed from transparent to the original black.

The sequence diagram for displaying the banner is the same as the sequence diagram for displaying the fault shown in FIG. 31. However, when displaying a banner, the user may want to operate an external application that is not hidden by the banner display.

FIG. 32 is a sequence diagram in a state where the banner is displayed. It is a sequence diagram in the case where the user operates the external application part other than the banner display in the state where the external application is started, the window of the external application is displayed, and the banner is displayed.

When the user operates the part other than the banner display where the external application is visible, the event transmission / reception unit 311 of the framework 31 uses the transparent div element corresponding to the external application window area to send the user's operation event. Upon acceptance, the device service layer 32 is requested to transfer the operation event.

The device service layer 32 transfers the operation event to the external application in response to the transfer request from the framework 31. As a result, the external application accepts the user's operation and executes the process according to the operation. Since the banner display part is arranged in the hierarchy above the transparent div element, the operation event is not transferred to the external application via the transparent div element. The transparent div element as a dummy has a function to seamlessly manage and control the internal application window and the external application window, and even when the banner is superimposed on the external application, the part of the external application that is not hidden by the banner display It can be said that it has a function that enables operation.

FIG. 33 schematically shows a transfer process of a user operation event in the banner display. Even if the banner is displayed, the user can see the external application through the transparent div element and the transparent background color layer 88. When the user operates the screen, the operation is detected as an operation for the transparent div element, and the framework 31 transfers the operation event to the external application via the device service layer 32. It can be said that the user is operating the external application through the transparent div element and the transparent background color layer 88.

The "component" in this embodiment means a logically separable component of software. The component can be executed by one or more processors. In this embodiment, JavaScript is used, but of course other programming languages can also be used.

Further, the present invention is not limited to the above embodiment, and various modifications are possible. A modified example will be described below.

<Modification example 1>
In the embodiment, the control unit (processor) 22 of the image processing device 12 executes the framework 31 of the presentation layer 30 and various applications 50. As shown in FIG. 2, the presentation layer 30 and the device service layer Since 32 is separated, the framework of the presentation layer 30 and various applications 50 are separated from the image processing device 12, for example, a processor of a mobile terminal such as a smartphone or a tablet terminal for controlling the image processing device 12. It may be configured to be executed. Further, it is desirable that the operation unit 26 in FIG. 1 is also mounted on the mobile terminal. In this case, the mobile terminal and the image processing device 12 can be collectively referred to as an image processing device or an image processing system.

<Modification 2>
In the embodiment, the background color (background color) is made non-transparent such as black in consideration of the visual effect of the screen transition, the external application window is displayed, and the background color is transparent in the case of fault display or banner display (the background color). (Including translucent) is set, but the background color may be changed according to the type of fault or banner to be displayed. For example, in the case of fault display, the background color is semi-transparent, in the case of banner display, the background color is transparent, in the case of fault display, the background color is gray, or a specific banner display. In the case of, the background color is gray, and in the case of other banner displays, the background color is transparent. The purpose of making the background color transparent is to make the external application window visible to the user despite the fault display and banner display, so make the background color transparent only when it is necessary to make the external application visible. Can be set.

<Modification example 3>
In the embodiment, an application that is standardly implemented in a system that provides an I / F defined by the framework 31 is an internal application (internal application), and is an application that can be added and is provided by an existing application or a third party. An application that is an application and does not always provide the I / F defined by the framework 31 is defined as an external application (external application), but the application is not limited to this, and different types of first application and second application are defined. It can be widely applied to the system to be implemented. For example, in the case where a standard application or the like pre-installed in the system is used as the first application, other addable applications or the like are used as the second application, and the window (first window) of the first application is displayed together with the background color. Therefore, when the window of the second application (second window) is displayed in the hierarchy below the first window of the first application, the background color can be set to transparent (including semi-transparent).

10 terminal equipment, 12 image processing equipment, 16 ROM, 18 RAM, 20 HDD, 24 input / output interface I / F, 26 operation unit, 28 image processing unit, 30 presentation layer, 31 framework, 32 device service layer, 50 applications , 52 Core Logic, 54 UI frame, 56 Manifest File, 74, 75, 76, 78 Companion App, 88 Ground Color (Background Color) Layers.

Claims (7)

The first and second applications that run on the framework,
A control means for executing the first application, the second application, and the framework, and
By executing the framework, the control means displays the first window of the first application together with the background color, and displays the second window of the second application in the hierarchy below the first window. If you want to set the background color to transparent ,
The control means is an image processing device that sets the background color to be transparent when a fault display or a banner is displayed in the first window of the first application. The image processing apparatus according to claim 1, wherein the control means sets the background color to be non-transparent when the second window of the second application is not displayed. The first and second applications that run on the framework,
A control means for executing the first application, the second application, and the framework, and
By executing the framework, the control means displays the first window of the first application together with the background color, and displays the second window of the second application in the hierarchy below the first window. If you want to set the background color to transparent,
The control means embeds a dummy management element for managing the window of the second application in the first window of the first application in response to an instruction to start the second application, and the second application. An image processing device that displays the second window of the above and transparently displays the dummy management element. The image processing device according to claim 3, wherein the control means executes the second application with an operation on the transparently displayed dummy management element as an operation on the second application. The image processing apparatus according to any one of claims 1 to 4, wherein the first application is a pre-embedded application, and the second application is an addable application. The first application is separated into a core logic unit that handles basic processing and a UI frame unit that handles drawing processing, and the core logic unit implements an interface (I / F) defined by the framework according to claim 5. The image processing apparatus described. A program that is executed by the processor that controls the image processing device.
The processor is made to function as a first application and a second application running on the framework and a control means for executing the framework.
A step of displaying the first window of the first application together with the background color by executing the framework, and
A step of setting the background color to be transparent when the second window of the second application is displayed in the hierarchy below the first window, and
A step of setting the background color to be transparent when a fault display or a banner is displayed in the first window of the first application.
A program that executes.

Images