JP6488608B2 - Information processing apparatus, system, and program - Google Patents

Description

  The present invention relates to an information processing apparatus, system, and program.

  Patent Document 1 describes a technique for displaying display screen data as an operation panel on an operation unit of an image forming apparatus and a display unit of an external information terminal.

JP2013-256016A

  The present invention provides a system in which a series of processing is executed between a server side and a client side in combination with a plurality of communication protocols, regardless of the order of reception of the series of data processing on the client side. The purpose is to prevent the change.

An information processing apparatus according to claim 1 of the present invention includes software that functions as a client, first establishing means for establishing a first communication path based on a request / response type first communication protocol, the software, and the first A second establishing means for establishing a second communication path according to a second communication protocol different from the communication protocol; a receiving means for receiving a processing execution request from the software via the first communication path; and the received execution request. According to the process execution means, a first message indicating that the execution request has been received, and a second message indicating the processing status of the process, a transaction number identifying the process , and the first message And a sequence indicating the processing order of each message in the message group including the second message And assigning means for assigning items, and a first transmission means for transmitting to said software by said first communication channel without the first message which the transaction number and the sequence number is assigned, to wait for completion of the process, A second transmission unit configured to transmit the second message to which the transaction number and the sequence number are assigned to the software through the second communication path during execution of the process and when the process is completed; And

An information processing apparatus according to a second aspect of the present invention includes software that functions as a server, first establishing means for establishing a first communication path based on a request / response type first communication protocol, the software, and the first A second establishing means for establishing a second communication path according to a second communication protocol different from the communication protocol; a first transmitting means for transmitting a processing execution request to the software via the first communication path; A first receiving means for receiving a first message indicating that the execution request has been accepted via the first communication path ; a processing status during execution of the process from the software via the second communication path; and the process a second receiving means for receiving a second message indicating completion, second message first message and the reception said received Based on a transaction number for identifying a process corresponding to the transmitted execution request and a sequence number indicating a processing order of each message in a message group including the first message and the second message. , Specifying means for specifying the processing order of the first message and the second message, and processing corresponding to each of the received first message and the received second message in the specified processing order And a processing execution means for executing the processing.

  An information processing apparatus according to a third aspect of the present invention is the information processing apparatus according to the second aspect, wherein the processing execution means has a processing order after the first message or the second message not received from the software. The process corresponding to the first message and the second message is not executed.

  The information processing apparatus according to claim 4 of the present invention is the information processing apparatus according to claim 3, wherein the processing execution means is not received even after waiting for a predetermined period or more. When the message is not received, processing corresponding to the first message and the second message in the processing order after the first message or the second message not received is executed.

  An information processing apparatus according to claim 5 of the present invention is the configuration according to claim 2, wherein the processing execution means has executed the processing order corresponding to the received first message or second message. If the processing order is earlier than the processing order, processing corresponding to the received first message or second message is not executed.

  An information processing apparatus according to claim 6 of the present invention is the configuration according to any one of claims 2 to 5, wherein the process execution means has received the first message that has not yet been executed. When there are a plurality of at least one of the second message and the second message, the attribute of the first message and the second message satisfying a predetermined attribute of at least one of the plurality of the first message and the second message. A process corresponding to at least one of them is executed.

A system according to a seventh aspect of the present invention includes a first information processing device and a second information processing device, wherein the first information processing device includes first software that functions as a client, and a request / response type First establishing means for establishing a first communication path according to a first communication protocol, the first software, second establishing means for establishing a second communication path according to a second communication protocol different from the first communication protocol, and A first receiving unit configured to receive a process execution request from the first software via the first communication path; a first process executing unit configured to execute the process according to the received execution request; and the execution request received the first message and a second message indicating the processing status of the processing indicating the fact, transaction number for identifying the process, as well as, Contact the first message And assigning means for assigning a sequence number indicating the processing order of each message in the message group including fine the second message, the first message that the transaction number and the sequence number is assigned, to wait for the completion of the process a first transmission means for not transmitting by said first communication path to the first software, when running and the processing of the processing is complete, the second message the transaction number and the sequence number is assigned and a second transmission means for transmitting to said first software by the second communication path, the second information processing apparatus, a second software functioning as a server, establishing the first communication path by the first communication protocol Different from the first establishing protocol, the second software, and the first communication protocol. Fourth establishing means for establishing the second communication path according to the second communication protocol that a third transmission means for transmitting the processing execution request via the first communication path to the second software, the second Second receiving means for receiving a first message indicating that the execution request has been received from the software via the first communication path; and the execution request transmitted from the second software via the second communication path. And a third receiving means for receiving a second message indicating the processing status during the execution of the processing corresponding to the processing and the completion of the processing, and each of the received first message and the received second message. A transaction number for identifying a process corresponding to the transmitted execution request, and a message group including the first message and the second message. Specific means for specifying the processing order of the first message and the second message based on a sequence number indicating the processing order of each message, and each of the received first message and the received second message And a second process execution means for executing a process corresponding to the above in the specified process order.

According to an eighth aspect of the present invention, there is provided a program for causing a computer to function as a client, establishing a first communication path using a request / response type first communication protocol, the software, and the first communication protocol. Establishing a second communication path based on a second communication protocol different from the above, receiving a process execution request from the software via the first communication path, and executing the process according to the received execution request a step, the second message indicating the processing status of the execution request a first message and said processing indicating the accepted transaction number for identifying the process, as well as a message including the first message and the second message Sequence that indicates the processing order of each message in the group A step of imparting number, and sending a first message to the transaction number and the sequence number is assigned, to the software by the first communication path without waiting for completion of the processing, execution of the process And a step of transmitting a second message to which the transaction number and the sequence number are assigned to the software through the second communication path when the processing is completed. To do.

According to a ninth aspect of the present invention, there is provided a program for causing a computer to function as a server, to establish a first communication path based on a request / response type first communication protocol, the software, and the first communication protocol. Establishing a second communication path according to a second communication protocol different from the above, transmitting a process execution request to the software via the first communication path, and from the software via the first communication path Receiving a first message indicating that an execution request has been received, and receiving a second message indicating a processing status during execution of the process and completion of the process from the software via the second communication path Each of the received first message and the received second message A transaction number for identifying a process corresponding to the transmitted execution request, and a sequence number indicating a processing order of each message in the message group including the first message and the second message, A step of specifying a processing order of the first message and the second message, and a step of executing processing corresponding to each of the received first message and the received second message in the specified processing order It is the program for performing these.

According to the first, second, third, seventh, eighth, and ninth aspects of the present invention, in a system in which a series of processes are executed using a plurality of communication protocols between the server side and the client side, the client side It is possible to prevent the processing order from being changed regardless of the reception order of the series of processing of the data.
According to the fourth aspect of the present invention, it is possible to prevent the processing from being stopped by the client side software.
According to the fifth aspect of the present invention, compared to the case where all the processes corresponding to the received message are executed, the load associated with the execution of the process when the message reception order is switched in the client-side software is increased. Reduce.
According to the invention of claim 6, the load related to the execution of the process is reduced as compared with the case where all the processes corresponding to the received message are executed.

1 is a block diagram showing an example of the overall configuration of an information processing system according to an embodiment The figure which shows an example of the displayed operation image The figure which shows the example of the status information which is displayed The block diagram which shows an example of the hardware constitutions of a user apparatus Block diagram showing an example of the hardware configuration of the operating device Block diagram showing an example of a hardware configuration of an information processing apparatus Block diagram showing an example of a functional configuration of an information processing system The figure which shows an example of the hierarchical structure showing the structure of an information processing system The figure which shows an example of the module group contained in a function part The figure which shows the example of the status information which is displayed The figure which shows the example of the status information which is displayed The sequence diagram which shows an example of operation | movement of each apparatus in case the 1st method is used The sequence diagram which shows an example of operation | movement of each apparatus in case the 2nd method is used The sequence diagram which shows an example of operation | movement of each apparatus in case the 3rd method is used The sequence diagram which shows an example of operation | movement of each apparatus in case the 4th method is used Sequence diagram showing an example of copy operation The figure which shows an example of the hardware constitutions of the information processing apparatus of a modification The figure which shows an example of the hardware constitutions of the information processing apparatus of a modification

[1] Embodiment First, the overall configuration, hardware configuration, and functional configuration of an information processing system according to an embodiment will be described with reference to FIGS. 1 to 11.

[1-1] Overall Configuration FIG. 1 is a block diagram illustrating an example of the overall configuration of the information processing system. In FIG. 1, an information processing system 1 including a communication line 2, a communication device 3, an information processing device 10, and a plurality of user devices 20 is illustrated. In the present embodiment, the information processing system 1 is a system that provides a user with functions such as copying, scanning, FAX (facsimile), and printing (output of image data to a medium).

  The communication line 2 is a system for communication including at least one of the Internet, a mobile communication network, a telephone line, and the like. The communication line 2 mediates communication between devices connected to the own system. An information processing apparatus 10 and a communication device 3 are connected to the communication line 2. The communication device 3 is a device having a communication function, and performs wireless communication in accordance with a wireless LAN standard in the present embodiment. The communication device 3 communicates with the information processing device 10 via the communication line 2 while performing wireless communication with the user device 20. That is, the information processing apparatus 10 communicates with the user apparatus 20 via the communication line 2 and the communication device 3. Further, the information processing apparatus 10 and the user apparatus 30 may be configured to communicate directly without using the communication device 3.

  The information processing apparatus 10 is an apparatus that performs processes such as an image forming process for forming an image on a medium and an image reading process for reading an image formed on the medium. These processes are performed when the above-described functions such as copying, scanning, FAX, and printing are provided to the user. The information processing apparatus 10 includes an operation device 30 that is used to operate the own device. The operation device 30 is a device for operating the information processing apparatus 10 and is a so-called control panel. The operation device 30 is fixed to the housing of the information processing device 10 and is used by a user who has come to a place where the information processing device 10 is installed. The operation device 30 includes, for example, a liquid crystal display and displays image data (hereinafter referred to as “operation image data”) for operating the information processing device 10. The operation image displayed based on the operation image data will be described with reference to FIG.

  FIG. 2 is a diagram illustrating an example of an operation image displayed on the operation device 30. In the example of FIG. 2, an operation image data group A1 for using the copy function is displayed. The operation image data group A1 includes operation image data A11 for starting copying. The user performs an operation corresponding to the operation image data A11, that is, an operation to start copying, by tapping (tapping) the operation image data A11. When an operation is performed on the displayed operation image data, the operation device 30 requests the information processing device 10 for processing corresponding to the operation. If the operation image data A11 is operated, the controller device 30 performs a process for providing a copy function (specifically, a series of processes including an image reading process and an image forming process; hereinafter referred to as “copy process”). ) To the information processing apparatus 10.

  Returning to the description of FIG. The user device 20 is a device such as a tablet terminal or a personal computer. The user device 20 may be carried and used by a user, or may be installed on a desk and used. In addition, the user device 20 is not limited to the operation of the information processing device 10 and may be used for other purposes (such as company business). The user device 20 displays the operation image data, and when an operation is performed on the displayed operation image data, the user device 20 makes a request for processing corresponding to the operation to the information processing device 10. And in common.

  When the information processing apparatus 10 executes a process based on a request from the operation apparatus 30 or the user apparatus 20, the information processing apparatus 10 transmits information indicating the status of the process being executed (hereinafter referred to as “situation information”) to these apparatuses. Then, the processing status is notified to these devices. The status information is information indicating the status of processing such as the number of media on which an image is read and the number of media on which an image is formed, for example, when the copy function is used.

  FIG. 3 is a diagram illustrating an example of the displayed status information. For example, in the case of executing a copy process, the information processing apparatus 10 copies the number information C11 indicating the number of copies (number of copies and pages) as shown in FIG. ”And status information C1 including copy parameters such as“ monochrome ”,“ A4 ”,“ 100% ”, and“ sort ”.

  In addition, when the copy process proceeds and the requested number of copies is completed, the information processing apparatus 10, as shown in FIG. 3B, the number information C21 indicating that the requested number of copies has been completed. And status information C2 including a character string such as “Copy completed” is transmitted. Note that the number information C11 and C21 are expressed in fractions with the number of copies requested as a denominator and the number of copies completed as a numerator. As described above, the status information includes information indicating the progress of the process, information indicating the result of the process, information on the parameters used for the process, and the like. The operation device 30 and the user device 20 are both display devices that display the notified status as shown in FIG. Hereinafter, when the operation device 30 and the user device 20 are not distinguished, they are referred to as the display device 4.

[1-2] Hardware Configuration FIG. 4 is a block diagram illustrating an example of a hardware configuration of the user device 20. The user device 20 includes a control unit 21, a storage unit 22, a communication unit 23, a display unit 24, and an operation unit 25. The control unit 21 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and a real time clock (all not shown), and the CPU uses the RAM as a work area to store the ROM and the memory. The operation of each unit is controlled by executing a program stored in the unit 22. The real time clock calculates the current date and time and notifies the CPU.

  The storage unit 22 includes a hard disk drive and the like, and stores data, programs, image data, and the like used by the control unit 21 for control. The communication unit 23 is a communication unit including a communication circuit and an antenna that perform wireless communication in accordance with a wireless LAN (Local Area Network) standard. For example, the communication unit 23 performs wireless communication with the communication device 3 illustrated in FIG. The display unit 24 includes, for example, a liquid crystal display and is controlled by the control unit 21 to display an image on the display surface. For example, when the device itself is a tablet terminal, the operation unit 25 includes a touch sensor (also referred to as a touch screen or a touch panel) provided on the display surface, a button provided on the housing, and the like described above. The operation data indicating the content of the operation is supplied to the control unit 21. In the case where the own apparatus is a personal computer, the operation unit 25 may include a keyboard and a mouse. The control unit 21 performs control according to the supplied operation data.

  FIG. 5 is a block diagram illustrating an example of a hardware configuration of the controller device 30. The operation device 30 includes a control unit 31, a storage unit 32, a communication unit 33, a display unit 34, and an operation unit 35. In the following, blocks representing hardware common to the blocks shown in FIG. 4 (each unit other than the communication unit 33) are assigned common names and description thereof is omitted. The communication unit 33 is a communication unit that includes a communication circuit that performs communication in accordance with the standard of the wired LAN, and a port into which a connector included in a communication cable (specifically, a LAN cable) is inserted.

  As described above, when the operation device 30 and the user device 20 are not distinguished, they are referred to as the display device 4. Accordingly, when it is not necessary to distinguish between the control unit 31 and the control unit 21, the control unit 31 is referred to as a “control unit 141”. When it is not necessary to distinguish between the storage unit 32 and the storage unit 22, the storage unit 32 is referred to as a “storage unit 142”. When there is no need to distinguish between the display unit 34 and the display unit 24, the display unit 34 is referred to as a “display unit 144”. When there is no need to distinguish between the operation unit 35 and the operation unit 25, the operation unit 35 is referred to as “operation unit 145”.

  FIG. 6 is a block diagram illustrating an example of a hardware configuration of the information processing apparatus 10. The information processing apparatus 10 includes a control unit 11, a storage unit 12, an image reading unit 13, an image forming unit 14, a first communication unit 15, a second communication unit 16, and a connection unit 17. Hereinafter, blocks (control unit 11 and storage unit 12) representing hardware common to the blocks illustrated in FIGS. 4 and 5 are denoted by common names and description thereof is omitted. The information processing apparatus 10 includes the above-described operation device 30 that functions as a UI (User Interface) unit. In the present embodiment, the control unit 11 controls each unit other than the operation device 30.

  The image reading unit 13 performs an image reading process for reading an image drawn on a document using, for example, a CCD (Charge Coupled Devices) system. The image reading unit 13 optically reads an image of content formed on a medium such as paper and supplies image data representing the read image to the control unit 11. The image forming unit 14 performs an image forming process for forming an image on a medium using, for example, an electrophotographic method. The image forming unit 14 forms an image indicated by the image data supplied from the control unit 11 on a medium such as paper. Note that the above-described methods used for image reading and image formation are examples, and other methods may be used.

  The first communication unit 15 is connected to the communication unit 33 of the controller device 30 via a communication cable or data bus, and communicates with the controller device 30 without passing through the external device described above. That is, the communication unit 33 also communicates with the information processing apparatus 10 without using the external device described above.

  The second communication unit 16 conforms to a communication circuit that performs communication in accordance with a wired LAN or wireless LAN standard as a communication interface, a port into which a connector of a communication cable (LAN cable) is inserted, or a wireless LAN standard. A compliant wireless transmission / reception device is provided, and a communication process for communicating with a device connected via this interface is performed. The second communication unit 16 is connected to the communication line 2 illustrated in FIG. 1 and communicates with, for example, the user device 20 via an external device (an external device for the information processing device 10, for example, the communication device 3). The connection unit 17 has a slot and the like for connecting storage media such as an SD memory card, and is connected to those storage media. The control unit 11 reads data stored in the storage medium and writes data to the storage medium via the connection unit 17. The operating device 30 has the configuration described in the description of FIG. 5 and communicates with the first communication unit 15. For example, the operation device 30 displays an image indicated by data transmitted from the control unit 11 or transmits data for operating the information processing device 10 to the control unit 11 by performing this communication. It functions as an interface unit (UI unit).

[1-3] Functional Configuration FIG. 7 is a block diagram illustrating an example of a functional configuration of the information processing system 1. The information processing system 1 includes an information processing device 10 (an example of a first information processing device) and a display device 4 (that is, an operation device 30 or a user device 20 (an example of a second information processing device)). The information processing apparatus 10 includes a first establishment unit 101, a second establishment unit 102, a first reception unit 103, a first process execution unit 104, a grant unit 105, a first transmission unit 106, and a second transmission. Means 107. The first establishment unit 101 establishes a first communication path using software that functions as a web client and a first communication protocol. The second establishment unit 102 establishes a second communication path based on a second communication protocol different from the first communication protocol. The first receiving unit 103 receives a processing execution request from the software via the first communication path established by the first establishing unit 101. The first process execution unit 104 executes a process according to the execution request received by the first reception unit 103. The granting unit 105 adds a processing order of each message to a first message indicating that the execution request received by the receiving unit 103 has been received and a second message indicating the processing status of the processing executed by the processing executing unit 104. Is assigned a sequence number. The first transmission means 106 transmits the first message assigned the sequence number by the assignment means 105 to the software via the first communication path. The second transmitting means 107 transmits the second message assigned the sequence number by the assigning means 105 to the software via the second communication path.

  In this example, the control unit 11 or the control unit 11 and the first communication unit 15 of the information processing apparatus 10 include a first establishment unit 101, a second establishment unit 102, a first reception unit 103, and a first process execution unit 104. , Giving means 105, first transmission means 106, and second transmission means 107.

  The display device 4 includes a third establishing unit 401, a fourth establishing unit 402, a third transmitting unit 403, a second receiving unit 404, a third receiving unit 405, a specifying unit 406, and a second process executing unit. 407. The third establishment unit 401 establishes a first communication path using software that functions as a web server and a first communication protocol. The fourth establishment unit 402 establishes a second communication path with the software using a second communication protocol different from the first communication protocol. The third transmission unit 403 transmits a process execution request to the software via the first communication path established by the third establishment unit 401. The second receiving unit 404 receives a first message indicating that the execution request transmitted by the third transmitting unit 403 has been received from the software via the first communication path. The third receiving means 405 receives a second message indicating the processing status of processing from the software via the second communication path. The specifying unit 406 determines the first message and the second message based on the sequence number given to each of the first message received by the second receiving unit 404 and the second message received by the third receiving unit 405. The processing order of the processing corresponding to is specified. The second process execution means 407 executes the processes corresponding to the received first message and second message in the process order specified by the specifying means 406.

  In this example, the control unit 21, the control unit 21 and the communication unit 23 of the user device 20, the control unit 31 of the operating device 30, or the control unit 31 and the communication unit 33 include a third establishment unit 401, a fourth establishment unit 402, This is an example of the third transmitting unit 403, the second receiving unit 404, the third receiving unit 405, the specifying unit 406, and the second processing execution unit 407.

[1-4] Hierarchical Structure FIG. 8 is a diagram illustrating an example of a hierarchical structure representing the configuration of the information processing system 1. The information processing system 1 includes a presentation layer 40, an interface 50, a function layer 60, a middleware layer 70, and a hardware layer 80. The presentation layer 40 is realized by the control unit 141 of the display device 4 (that is, the operation device 30 and the user device 20) executing a program stored in the storage unit 142. The presentation layer 40 is an example of software that functions as a web client. The interface 50, the function layer 60, and the middleware layer 70 are realized when the control unit 11 of the information processing apparatus 10 executes a program stored in the storage unit 12. At least one of the interface 50, the function layer 60, and the middleware layer 70 is an example of software that functions as a web server. The hardware layer 80 is a set of hardware resources of the information processing apparatus 10 and is controlled by the middleware layer 70.

  In this embodiment, the presentation layer 40 (that is, the display device 4) and the interface 50 (that is, the information processing device 10) communicate according to the WWW (World Wide Web: hereinafter referred to as “web”) standard. Is done. In this example, the presentation layer 40 and the interface 50 communicate with each other using HTTP (Hypertext Transfer Protocol) and WebSocket.

  When the display device 4 is the user device 20, the information processing device 10 and the user device 20 (that is, the presentation layer 40 and the interface 50) are connected via the communication line 2 and the communication device 3 as described in the description of FIG. Send and receive data. That is, in this case, communication is performed via a communication device (such as the communication device 3 or the communication line 2 included in the communication line 2; hereinafter referred to as “other communication device”) that is not the information processing device 10 or the user device 20. On the other hand, when the display device 4 is the operation device 30, the information processing device 10 and the operation device 30 (that is, the presentation layer 40 and the interface 50) are connected to the first communication unit 15 described in FIG. Send and receive data via. That is, in this case, communication is performed without using the above-described other communication device. In addition to the communication device 3, the other communication device includes, for example, a router, a hub, a bridge, and the like included in the communication line 2.

[1-4-1] Presentation Layer The presentation layer 40 realized by the operation device 30 and the user device 20 realizes a function (so-called user interface) that receives an operation by the user. The presentation layer 40 includes a UI control unit 41, a device control unit 42, and a communication unit 43. The UI control unit 41 is a part that controls the user interface, and displays the above-described operation image and status information transmitted from the interface 50 (that is, the information processing apparatus 10) on the display unit 144 or from the operation unit 145. A user operation is accepted according to the supplied data. Specifically, for example, the UI control unit 41 displays the situation information C1 and the situation information C2 shown in FIG. Here, another example of the situation information will be described with reference to FIGS.

  FIG. 10 is a diagram illustrating an example of the displayed status information. In this example, the status information displayed when the storage unit for storing the medium is opened by the user is displayed. Specifically, status information C3 including a character string “No paper storage unit is set” is displayed.

  FIG. 11 is a diagram illustrating another example of the displayed status information. In this example, when a storage medium such as an SD memory card storing a plurality of images is connected to the connection unit 17 of the information processing apparatus 10 shown in FIG. 6, a list of thumbnail images of the images stored in the storage medium. The status and result of the process for displaying are displayed. In FIG. 11A, status information C4 including a character string “image list is displayed” and thumbnail images (10 thumbnail images in this example) generated from the read-in images is shown. Has been. FIG. 11B shows status information C5 including a character string “Image list displayed” and a list of thumbnail images generated from all images read from the storage medium.

  In addition, the UI control unit 41 transmits a request for operation image data to the interface 50 of the information processing apparatus 10 when the user performs an operation to access a URL (Uniform Resource Locator) of the information processing apparatus 10, for example. The UI control unit 41 displays the operation image indicated by the operation image data transmitted from the operation image management unit 63 of the information processing apparatus 10 on the display unit 144 as a response to the request for the transmitted operation image data. More specifically, the UI control unit 41 displays an operation image based on parameters such as the arrangement and size of the operation image indicated by the operation image data. The operation image displayed in this way is, for example, the operation image data group A1 shown in FIG. An example of information constituting the operation image data is a file described in an HTML (Hypertext markup language) format. The UI control unit 41 controls to interpret the acquired HTML file and display an operation image or accept operation information.

  The operation image data group A1 includes operation images (for example, button images and icon images) for selecting parameters such as the magnification of the document image to be copied, the paper on which the copied document image is formed, and the color mode of the copied document image. include. The UI control unit 41 displays an image in a different display manner in order to present to the user the contents of parameters that are selected or set by the user operating these operation images. In the example of FIG. 2, an image including a character string representing the magnification of the currently selected copy (in this example, “110%”) and an operation image representing the currently selected paper and color mode (in this example, “ A different aspect from the selected image that has not been selected (an aspect in which the character strings “other magnification”, “A4 plain paper”, and “black and white” are drawn) (in this example, a line is thickened. Highlight or radio button) Or the like.) Is an example displayed.

  The operation image data group A1 includes operation image data A11 for starting copying. When the operation image data A11 is operated, a copy process using the parameter selected at that time is requested to the function layer 60 via the interface 50.

  Returning to the description of FIG. The UI control unit 41 supplies data related to the displayed operation image to the device control unit 42. The data related to the operation image is data used to determine whether or not the operation image has been operated, and is identification information of the currently displayed screen or parameters selected or set by the user on the operation screen. Data such as type and value.

  The device control unit 42 requests the function layer 60 to execute processing via the interface 50 based on a user operation. When an operation on the operation image data is performed by the user, the device control unit 42 requests the information processing apparatus 10 to perform processing corresponding to the user operation based on the operation data supplied from the UI control unit 41.

  The device control unit 42 uses the parameters supplied from the UI control unit 41 to generate, as instruction data, the content requested to be processed by the information processing apparatus 10, and transmits the instruction data to process the information processing apparatus 10. To request. If the operation image data A11 is operated in the state shown in FIG. 2, the UI control unit 41 sets the information indicating the copy processing request, “11%” as the copy magnification parameter, and the color mode parameter. “Monochrome” and “A4 plain paper” as a paper size parameter are sent to the device control unit 42, and the device control unit 142 generates instruction data representing a process for copying to A4 plain paper in black and white at a magnification of 110%. To the information processing apparatus 10.

  In this example, the device control unit 42 generates instruction data describing an instruction to be performed using instruction information and parameters related to the function received from the UI control unit 42. In this embodiment, the instruction data is described in XML (Extensible Markup Language). The device control unit 42 generates an HTTP request including the generated instruction data. The device control unit 42 requests the function layer 60 to perform processing by transmitting the generated HTTP request to the interface 50. The HTTP request that the device control unit 42 transmits to the interface 50 in order to request processing is hereinafter referred to as “processing request”. When the copy process is requested, a process request including instruction data is generated. However, depending on the requested process, the instruction data may not be included in the process request. For example, when a process for turning off the power of the information processing apparatus 10 is requested, the instruction data is not included in the process request.

  The communication unit 43 controls communication performed by the UI control unit 41 and the device control unit 42 with the interface 50.

[1-4-2] Interface 50
The interface 50 is a layer that realizes a function that mediates the connection between the presentation layer 40 and the function layer 60. The connection between the presentation layer 40 and the interface 50 is through an external device having a communication function such as the communication device 3 shown in FIG. 1 (connection with the user device 20) or not through such an external device (operation device 30). Connection). The interface 50 mediates communication with the user device 20 and the operation device 30 by a common communication procedure (communication procedure compliant with standards such as HTTP and WebSocket). The interface 50 receives, for example, an HTTP request transmitted from the presentation layer 40 and supplies it to the function layer 60, or receives an HTTP response supplied from the function layer 60 as a response to the HTTP request and transmits it to the presentation layer 40. To do.

[1-4-3] Function layer 60
The function layer 60 is a layer that realizes a function of processing data in accordance with a target application such as a copy function and a scan function. The function layer 60 includes a reception / response unit 61, an event notification unit 62, an operation image management unit 63, an operation image DB (Data Base) 64, a function unit 65 that provides a copy function, a scan function, and the like. Is provided.

  The reception / response unit 61 receives a processing request transmitted from the presentation layer 40 via the interface 50 and requests the function unit 65 to perform processing. At this time, the reception / response unit 61 issues a transaction ID as information for identifying the requested process. Further, the reception / response unit 61 issues a client ID as information for identifying the request source (display device 4) that has requested the processing. The reception / response unit 61 stores the issued transaction ID and client ID and the communication destination of the request source display device 4 in association with each other in the storage unit 12. The reception / response unit 61 supplies the issued transaction ID to the functional unit 65 when requesting the processing to the functional unit 65. Further, the reception / response unit 61 transmits a response to the received request to the presentation layer 40 via the interface 50. The response may only indicate that the request has been accepted, or it may include status information.

  The event notification unit 62 generates notification data including situation information, includes the generated notification data in data conforming to WebSocket, and transmits the data to the presentation layer 40 via the interface 50.

  In the present embodiment, the function layer 60 and the presentation layer 40 roughly transmit the request, response, or notification by four methods. The first and second methods are methods using the HTTP protocol, and these are performed by the reception / response unit 61. On the other hand, the third method is a method in which the HTTP protocol and the WebSocket protocol are used together, and this method is performed by the cooperation of the reception / response unit 61 and the event notification unit 62. The fourth method is a method in which notification is transmitted by the WebSocket protocol, and this method is performed by the event notification unit 62.

  In the first method, when a processing request (HTTP request) is transmitted from the presentation layer 40, the reception / response unit 61 waits until execution of the processing by the middleware layer 70 and the hardware layer 80 is completed. In this case, when the completion / processing is notified from the middleware layer 70, the reception / response unit 61 transmits an HTTP response including status information indicating the status when the processing is completed to the presentation layer 40 that has requested the processing. To do. As a specific example, when the user operates an operation image such as “−” or “+” in the “magnification selection” column shown in FIG. Is required. In this case, the reception / response unit 61 transmits a response including the changed parameter value to the presentation layer 40 by executing the requested process.

  In the second method, when a processing request (HTTP request) is received from the presentation layer 40, the reception / response unit 61 first responds that the processing request has been received without waiting for completion of the requested processing ( (HTTP response). Thereafter, the status information is acquired by the presentation layer 40 performing so-called polling by HTTP. That is, when there is a request for notification of the processing status from the presentation layer 40 (hereinafter referred to as “notification request”), the reception / response unit 61 responds to the notification request. When a response is made by the second method, for example, the number information C11 shown in FIG. 3A is notified and displayed every time polling is performed, not every time a copied image is formed on a medium. For this reason, for example, the number of pages does not increase one by one, but may increase by several.

  As described above, when the first and second methods are used, the reception / response unit 61 transmits a response to the request when there is a request (processing request or notification request) from the presentation layer 40. When the first method is used, the reception / response unit 61 transmits the processing result to the requesting presentation layer 40 in response to the processing request, and when the second method is used, the processing request In response to the notification request received after the response to, the processing result and the status of other processing are transmitted to the presentation layer 40 of the request source.

  In the third method, when the processing request (HTTP request) is received from the presentation layer 40, the reception / response unit 61 first requests the processing without waiting for completion of the requested processing as in the second method. A response (HTTP response) indicating that has been accepted is transmitted. After that, when a situation to be notified occurs in the processing executed by the middleware layer 70 and the hardware layer 80, the event notification unit 62 includes a message (notification) including status information indicating the situation even if there is no notification request. Is transmitted to the presentation layer 40 by WebSocket. That is, the situation notified by the third method is the situation of the process executed by the process request. When notification is transmitted by the third method, for example, the “number of pages” in the sheet number information C11 shown in FIG. 3A increases every time a copied image is formed on a medium.

  In the fourth method, the event notification unit 62, when a situation to be notified occurs regardless of whether or not there is a processing request from the presentation layer 40, includes a message (notification) including status information indicating the situation even if there is no notification request. ) Is transmitted to the presentation layer 40 by WebSocket. That is, the situation notified by the fourth method has nothing to do with the process executed by the process request. In the fourth method, for example, any presentation layer 40 in which a session has been established with the information processing apparatus 10 based on the WebSocket standard can be a notification destination. In the information processing system 1, for example, statuses such as “out of toner”, “out of paper”, “opening of storage unit (a storage unit storing a medium such as paper) is opened” are notified by the fourth method. It is determined as follows. Note that the situation notified by the fourth method may be different for each presentation layer 40 (display device 4) that is a notification destination.

  As described above, when the third and fourth methods are used, the event notification unit 62 actively (so-called push type) regardless of whether or not there is a request (processing request and notification request) from the presentation layer 40. Make a notification. When the third method is used, the event notification unit 62 notifies the result of the processing and the status of other processing after the response to the processing request, and when the fourth method is used. Notifies the predetermined processing status.

  When the second and third methods are used, in either case, the reception / response unit 61 or the event notification unit 62 displays the processing result and other processing status after the response to the processing request. Send. The three methods from the first to the third are methods for notifying the status of the requested processing. In this embodiment, when the presentation layer 40 requests processing, the presentation layer 40 designates a method (that is, one of the first method to the third method) for transmitting the processing status (including the processing result). . For example, the transmission method may be specified by a parameter included in the processing request generated by the device control unit 42. The reception / response unit 61 and the event notification unit 62 transmit a response and / or notification using a method specified by the presentation layer 40.

  In this embodiment, the situation information is generated by the middleware layer 70 and supplied to the reception / response unit 61 or the event notification unit 62. The status information may be generated by the function unit 65, the reception / response unit 61, or the event notification unit 62. Further, the status information supplied to the event notification unit 62 is not supplied directly from the middleware layer 70 (or the function unit 65) to the event notification unit 62, but to the event notification unit 62 via the reception / response unit 61. It may be supplied.

  Returning to the description of FIG. The operation image management unit 63 manages operation image data to be transmitted to the presentation layer 40. The operation image DB 64 is a database that stores operation image data. The operation image management unit 63 provides the operation image data described above to the presentation layer 40 via the interface 50. The operation image management unit 63 stores operation image data described in, for example, HTML that constitutes an operation screen as a user interface. When an operation image request is transmitted via the interface 50, the operation image management unit 63 stores the operation image data. A response including this is transmitted to the request source via the interface 50.

  When an operation image request is transmitted from the user device 20, the operation image management unit 63 transmits operation image data to the user device 20 in response thereto. Further, when a request for operation image data is transmitted from the operation device 30, the operation image management unit 63 transmits the operation image data to the operation device 30 in response thereto. By transmitting the operation image data to the operation image data request sources (the user device 20 and the operation device 30) as described above, the operation image management unit 63 provides the operation image data to the request sources.

  The function unit 65 is a module group for realizing the function provided to the user by the information processing apparatus 10. When a processing request is transmitted from the presentation layer 40 via the interface 50, the functional unit 65 of the information processing apparatus 10 controls the middleware layer 70 to execute the processing requested by the processing request.

  FIG. 9 is a diagram illustrating an example of a module group included in the functional unit 65. The function unit 65 includes a copy function, a scan function, a FAX function, a print function, a destination table management function (a function for managing destination information), a device management function (a document set state of the image reading unit 13 and a medium of the image forming unit 14). , A function for managing the state of consumables, etc.), an authentication function, a confidential box function (a function for managing an electronic document stored in the information processing apparatus 10), a preview function, a download function (a function for controlling program updates) Modules for realizing a maintenance function (a function for maintaining hardware according to a request from a remote) and a diagnosis function (a function for diagnosing hardware) are included.

  For example, the functional unit 65 analyzes the instruction data included in the processing request, and controls the middleware layer 70 so that the processing is executed according to the content of the analyzed instruction. Specifically, the function unit 65 causes the designated process (for example, copy process) to be executed with the designated parameter. Thereby, the middleware layer 70 reads an image from a medium with an image quality and a reading method determined by a user operation, for example, and forms the read image in an output format, magnification, paper, and color mode determined by the user operation. To do.

  When the function unit 65 instructs the middleware layer 70 to execute the process, the function unit 65 supplies the transaction ID supplied from the reception / response unit 61 to the middleware layer 70. The middleware layer 70 supplies a transaction ID to the reception / response unit 61 together with the situation information described above.

  Further, the function unit 65 analyzes the description specifying the method (any one of the first to third methods) described in the processing request, and analyzes the method (that is, the method specified by the display device 4). Is transmitted to the reception / response unit 61. The reception / response unit 61 or the event notification unit 62 transmits the notification data describing the situation information supplied to the communication destination associated with the transaction ID supplied from the function unit 65 to the communication destination. Send according to As a result, the reception / response unit 61 or the event notification unit 62 transmits the processing status to the processing request source in a specified manner. The function unit 65 may analyze the notification destination designated by the reception / response unit 61 by simply supplying the processing request to the reception / response unit 61.

  The middleware layer 70 is a layer that executes general-purpose processing between the function layer 60 and the hardware layer 80. The hardware layer 80 is a layer that physically realizes processing such as image reading and image formation. In addition to the copy process described above, the middleware layer 70 and the hardware layer 80 execute various processes such as a scan process, a FAX process, and a print process (processes for providing a scan function, a FAX function, and a print function, respectively). . The copy process includes an image reading process by the image reading unit 13 and an image forming process by the image forming unit 14 shown in FIG. The scanning process includes an image reading process, and the FAX process includes a FAX communication process by the second communication unit 16. The print process includes a communication process and an image forming process by the first communication unit 15. In addition to the scan process and the FAX process, the data communication process using the second communication unit 16 and the write process for writing the data to the storage unit 12 and the stored data according to the data acquisition and output method Read processing from the storage unit 12 is included. When the middleware layer 70 and the hardware layer 80 execute the processing, the middleware layer 70 and the hardware layer 80 supply the above-described status information (information indicating the status of the processing being executed) to the function layer 60.

  At least one of the reception / response unit 61, the event notification unit 62, the operation image management unit 63, and the function unit 65 illustrated in FIG. 8 is the first establishment unit 101, the second establishment unit 102, and the first reception unit illustrated in FIG. 103, an example of the first process execution unit 104, a grant unit 105, a first transmission unit 106, and a second transmission unit 107. At least one of the UI control unit 41, the device control unit 42, and the communication unit 43 includes a third establishment unit 401, a fourth establishment unit 402, a third transmission unit 403, a second reception unit 404, and a third reception unit 405. The specifying unit 406 and the second processing execution unit 407 are examples.

[1-5] Operation [1-5-1] Status Information Transmission Operation First, an example of an operation performed by each device when status information is transmitted by the above-described first to fourth methods will be described with reference to FIG. This will be described with reference to FIGS.

[1-5-1-1] First Transmission Operation of Status Information FIG. 12 is a sequence diagram illustrating an example of the operation of each apparatus when the above-described first method is used as the transmission method of the status information. is there. The operation illustrated in FIG. 12 is started, for example, when the user performs an operation of changing parameters for copying (magnification, paper size, color mode, etc.) before copying. First, when the control unit 141 (presentation layer 40) of the display device 4 accepts an operation for changing a parameter (step S111), the information processing device 10 includes processing request data describing a request for changing the parameter in the HTTP request. (Step S112). When receiving the HTTP request, the reception / response unit 61 of the information processing apparatus 10 interprets the content of the received request (step S113), and executes the processing with the interpreted content in the middleware layer 70 of the information processing apparatus 10. (Step S114).

  The middleware layer 70 cooperates with the hardware layer 80 to execute an instructed process, that is, a parameter change (step S115), and supplies information representing the result to the reception / response unit 61 as status information ( Step S116). The reception / response unit 61 generates notification data describing the supplied situation information (parameter change result in this example) (step S117). The reception / response unit 61 transmits an HTTP response including the generated notification data to the display device 4 that is a processing request source (step S118). The control unit 141 of the display device 4 displays the status information included in the transmitted HTTP response on the display unit 144 of the own device (step S119).

[1-5-1-2] Second Transmission Operation of Status Information FIG. 13 is a sequence diagram illustrating an example of the operation of each device when the second method is used. The operation illustrated in FIG. 13 is started, for example, when the user sets a document on the image reading unit 13 of the information processing apparatus 10 and performs an operation to start copying on the display device 4. First, when receiving an operation for starting copying (step S121), the control unit 141 of the display device 4 creates instruction data corresponding to the received operation (step S122). The control unit 141 transmits a processing request (HTTP request) including the created instruction data to the information processing apparatus 10 (step S123). Upon receipt of the processing request via the interface 50 shown in FIG. 8, the reception / response unit 61 of the information processing apparatus 10 interprets the content of the instruction by analyzing the instruction data included in the received processing request (step S124).

  When the reception / response unit 61 instructs the middleware layer 70 based on the interpreted content (Step S125), the middleware layer 70 and the hardware layer 80 cooperate to perform processing based on the instructed content (copying). Process) is executed (step S126). After instructing the copy process, the reception / response unit 61 transmits an HTTP response indicating that the request for the copy process has been received to the display device 4 as a response to the HTTP request received in step S123 (step S131). The control unit 141 of the display device 4 displays the content corresponding to the received HTTP response on the display unit 144 of the own device (step S132). Note that the operation in step S131 may be performed before the operation in step S124 or S125.

  In the second method, the processing status notification and the processing completion notification for the processing request from the display device 4 are acquired by the display device 4 when the display device 4 performs a polling process using an HTTP request. After receiving a response to the copy processing request, the control unit 141 of the display device 4 performs a polling process by transmitting a notification request (HTTP request) for requesting notification of the copy processing status (step S141). If the copy process status information has not been supplied yet, the reception / response unit 61 of the information processing apparatus 10 transmits an HTTP response indicating that fact to the display apparatus 4 (step S142). The operations in steps S141 and S142 are repeated.

  When the middleware layer 70 of the information processing apparatus 10 outputs a copy (an image showing the read document) to the medium, the middleware layer 70 detects the situation (step S151) and accepts status information indicating that the copy has been output to the medium. -It supplies to the response part 61 (step S152). When receiving the notification request (HTTP request) after the situation information is supplied (step S141), the reception / response unit 61 generates notification data including the situation information (step S153), and sends an HTTP response to the display device 4. (Step S154). The control unit 141 of the display device 4 displays the content corresponding to the HTTP response on the display unit of its own device as in step S132 (step S155).

[1-5-1-3] Third Transmission Operation of Status Information FIG. 14 is a sequence diagram illustrating an example of the operation of each device when the third method is used. In the first method and the second method, data is exchanged between the display device 4 and the information processing device 10 by HTTP. On the other hand, in the third method, data is transmitted from the information processing apparatus 10 to the display apparatus 4 by the WebSocket protocol in addition to HTTP. Specifically, the status information acquired by polling in the second method is notified to the display device 4 as a WebSocket message in the third method. The second method and the third method realize one process by different methods, and both are selectively used.

  The operation shown in FIG. 14 is started when the user performs an operation to start copying as shown in FIG. First, when the control unit 141 of the display device 4 receives an operation to start copying (step S121), the control unit 141 performs processing for establishing a WebSocket session with the information processing device 10. That is, the control unit 141 transmits an HTTP request that is a WebSocket handshake request to the information processing apparatus 10. This HTTP request includes a client ID for identifying the display device 4.

  The interface 50 of the information processing device 10 establishes a session to the WebSocket based on the HTTP request received from the display device 4. The reception / response unit 61 transmits a WebSocket handshake response to the display device 4, whereby a WebSocket session (hereinafter simply referred to as “session”) is established between the display device 4 and the information processing device 10. (Step S160). Note that the timing at which a session is established is not limited to this timing. For example, the session establishment process may be performed at the timing when the user performs an operation of starting a web browser using the display device 4 and accessing a URL (Uniform Resource Locator) of the information processing apparatus 10.

  In addition, the reception / response unit 61 stores the client ID included in the received HTTP request in a predetermined area of the storage unit 12 (hereinafter referred to as “cache area 12a”). The client ID stored in the cache area 12a is deleted from the cache area 12a when the session is disconnected by normal processing. That is, the cache area 12a stores the client ID of the display device 4 that is establishing a session with the information processing device 10 and the client ID of the display device 4 that has been disconnected during the processing. The

  In the processing after step S161, the display device 4 and the information processing device 10 exchange messages using either HTTP or WebSocket protocol. Hereinafter, when there is no need to distinguish between a message transmitted and received by HTTP (an example of a first message) and a message transmitted by a WebSocket (an example of a second message), these are simply referred to as “messages”. explain. The reception / response unit 61 of the information processing apparatus 10 stores the message transmitted by the own apparatus in the cache area 12a together with the client ID, transaction ID, and sequence ID assigned to the message. The sequence ID is identification information for identifying the message. Further, the control unit 141 of the display device 4 stores the set of the transaction ID and the sequence ID assigned to the message received by the device itself in the cache region (hereinafter referred to as “cache region 142a”) of the device itself. The cache area 142a may be provided in the storage unit 142 or may be provided in the RAM. The set of IDs stored in the cache area 142a is referred to when the session is disconnected during the processing.

  Returning to the description of FIG. When the WebSocket session is established, the control unit 141 transmits an HTTP request including notification request data describing that the third method described above is applied to the information processing apparatus 10 (step S161). When receiving the HTTP request, the reception / response unit 61 of the information processing apparatus 10 transmits the contents of the application described in the notification request data included in the received HTTP request to the middleware layer 70 (step S162). The middleware layer 70 receives the transmitted application and performs settings for transmitting status information in the third method in the future on the display device 4 (step S163). For example, the middleware layer 70 turns on a flag indicating whether or not to perform notification by the third method.

  Next, the operations from steps S122 to S126 and the operations of steps S131 and S132 shown in FIG. 13 are performed. Subsequently, when the middleware layer 70 and the hardware layer 80 of the information processing apparatus 10 perform the operation of step S151 (detection of copy output), it is said that the copy is output to the medium because the flag is ON. The situation information indicating the situation is supplied to the event notification unit 62 instead of the reception / response unit 61 (step S172). The event notification unit 62 generates notification data describing the supplied status information (number of copies, etc.) (step S173), and transmits the generated data to the display device 4 via the WebSocket session established in step S160. (Step S174). The control unit 141 of the display device 4 displays the contents described in the received data (for example, the status information C1 shown in FIG. 3A) on the display unit 144 of the own device (step S175).

  When the middleware layer 70 and the hardware layer 80 complete the copy process (step S181), the middleware layer 70 and the hardware layer 80 supply status information indicating the fact to the event notification unit 62 (step S182). The event notification unit 62 generates notification data describing the status information (for example, notification data including a character string “Copy completed”) (step S183), and uses the generated notification data as a WebSocket message. The data is transmitted to the display device 4 (step S184). The control unit 141 of the display device 4 displays the content described in the notification data received in step S184 (for example, the status information C2 shown in FIG. 7B) on the display unit of the own device (step S185).

[1-5-1-4] Fourth Transmission Operation of Status Information FIG. 15 is a sequence diagram illustrating an example of the operation of each device when the fourth method is used. In the fourth method, status information is notified to the display device 4 by WebSocket. In this method, a WebSocket session established in advance, for example, the session established in step S160 in FIG. 14 is used. When a predetermined situation occurs in the information processing apparatus 10 (out of toner, out of paper, opening of a storage unit, etc.), the middleware layer 70 and the hardware layer 80 of the information processing apparatus 10 cooperate to determine the predetermined situation. Is detected (step S201), and status information indicating the detected status is supplied to the event notification unit 62 (step S202). The event notification unit 62 generates notification data describing the supplied status information (step S203), and transmits the generated notification data to the display device 4 via the interface 50 via the established WebSocket session. (Step S204). The control unit 141 of the display device 4 displays the status information described in the notification data received in step S204 on the display unit of the own device (step S205).

  As described above, a different situation may be notified for each display device 4. In that case, before step S201, the control unit 141 transmits notification data describing a situation to be notified to the own apparatus to the information processing apparatus 10 via the communication unit 43. The event notification unit 62 that has received it via the interface 50 stores the status to be notified in association with the destination of the display device 4. The event notification unit 62 notifies the situation when the situation information supplied in step S202 represents the stored situation to be notified.

[1-5-2] Operation example 1
FIG. 16 is a sequence diagram illustrating an example of a specific operation of the information processing system 1. In this example, an operation example in the case where the user U1 operates the display device 4 to cause the information processing device 10 to execute a copy process will be described. In this example, HTTP (an example of a first communication protocol) and WebSocket (an example of a second communication protocol) are used in combination as communication protocols for exchanging data between the display device 4 and the information processing device 10. In this example, user authentication is performed by HTTP basic authentication.

  First, the user U1 operates the display device 4 to activate a web browser and access the URL of the information processing device 10. In step S301, the control unit 141 of the display device 4 starts a web browser. In step S <b> 302, the control unit 141 establishes a TCP (Transmission Control Protocol) connection (an example of a first communication path) with the information processing apparatus 10. This TCP connection is used as a lower layer communication protocol for sending and receiving HTTP requests and responses.

  In step S303, the control unit 141 establishes a TCP connection with the information processing apparatus 10, and transmits an HTTP request, which is a WebSocket handshake request, via the established TCP connection. The interface 50 switches to the WebSocket based on the HTTP request received from the display device 4. The interface 50 performs a WebSocket handshake response, whereby a WebSocket session (an example of a second communication path) is established between the display device 4 and the information processing device 10.

  The user sets an original on the image reading unit 13 of the information processing apparatus 10 and performs an operation to start copying on the display device 4. When the control unit 141 of the display device 4 receives an operation to start copying (step S304), the control unit 141 sends an HTTP request including notification request data describing that the third method described above is applied. 10 (step S305). When receiving the HTTP request, the reception / response unit 61 of the information processing apparatus 10 transmits the contents of the application described in the notification request data included in the received HTTP request to the middleware layer 70 (step S306). The middleware layer 70 receives the transmitted application, and performs settings for transmitting state information in the third method in the future on the display device 4 (step S307). For example, the middleware layer 70 turns on a flag indicating whether or not to perform notification by the third method.

  Next, the control unit 141 creates instruction data corresponding to the accepted operation (step S308). The control unit 141 transmits a processing request (HTTP request) including the created instruction data to the information processing apparatus 10 (step S309). Upon receipt of the processing request via the interface 50 shown in FIG. 8, the reception / response unit 61 of the information processing apparatus 10 interprets the content of the instruction by analyzing the instruction data included in the received processing request (step S310).

  When the reception / response unit 61 instructs the middleware layer 70 to perform copy processing based on the interpreted content (step S311), the middleware layer 70 and the hardware layer 80 cooperate to perform processing based on the instructed content (copy Process) is executed (step S312). After instructing the copy process, the reception / response unit 61 generates a message indicating that the request for the copy process has been received (step S313). The reception / response unit 61 assigns a sequence ID (an example of a sequence number) and a transaction ID to the generated message (step S314). The reception / response unit 61 transmits an HTTP request including a message with an ID to the display device 4 (step S315). Note that the operations in steps S313 to S315 may be performed before the operations in step S310 or S311.

  The sequence ID is identification information for identifying individual messages, and a different sequence ID is assigned to each message to be transmitted. On the other hand, the transaction ID is identification information assigned in units of processing. The same transaction ID is attached to messages exchanged until one process is completed. In this example, as the sequence ID, a value incremented in the order in which the messages are transmitted is used for each of the plurality of messages. In this example, the sequence ID is assigned a serial number indicating the processing order regardless of whether the message to be transmitted is transmitted by HTTP or WebSocket. In this example, “01” is assigned as the sequence ID to the message included in the HTTP response transmitted in step S315.

  When the middleware layer 70 and the hardware layer 80 of the information processing apparatus 10 perform the operation in step S316 (detection of copy output), the flag is turned on, and thus a copy is output to the medium. Is supplied not to the reception / response unit 61 but to the event notification unit 62 (step S317). The event notification unit 62 generates notification data describing the supplied status information (number of copies, etc.) (step S318), and assigns a sequence ID and transaction ID to the generated data (message) (step S319). The transaction ID given to this message is the same value as the transaction ID given in step S314. On the other hand, a value indicating that the processing order is later than the value assigned in step S314 is set in the sequence ID given to the message in step S319. In this example, “02” is added as the sequence ID to the message to which the ID is assigned in step S319.

  The event notification unit 62 transmits the message assigned with the ID to the display device 4 via the WebSocket session established in step S303 (step S320).

  The control unit 141 of the display device 4 receives a message from the information processing device 10. Based on the transaction ID and sequence ID of the received message, the control unit 141 identifies the processing order of the processing corresponding to the received message (step S321). In this example, the process corresponding to the received message includes a status information display process corresponding to the received message. The control unit 141 executes processing corresponding to the received message in the specified processing order.

  In the example illustrated in FIG. 16, the operation when the message transmitted in step S320 is received by the display device 4 before the message transmitted in step S315 is received by the display device 4 is illustrated. Since messages exchanged between the display device 4 and the information processing device 10 pass through the communication network, the order in which the messages are transmitted may be switched with the order in which the messages are received.

  As described above, the sequence ID “02” is attached to the message transmitted in step S320. However, the display device 4 has not yet received the message with the sequence ID “01” that is earlier in processing order than this message at the timing of receiving this message. In this example, when there is a message that has not been received, the control unit 141 does not execute a process corresponding to a message in a processing order after the message until the message is received. That is, the control unit 141 receives the message with the sequence ID “02”, but has not yet received the message with the sequence ID “01”. Therefore, the transaction ID attached to the message with the sequence ID “02”. And a message with the same transaction ID and a sequence ID “01” is awaited (step S322).

  When the middleware layer 70 and the hardware layer 80 complete the copy process (step S323), the middleware layer 70 and the hardware layer 80 supply status information indicating that to the event notification unit 62 (step S324). The event notification unit 62 generates notification data describing the situation information (for example, data including a character string “copying is completed”) (step S325), and assigns a sequence ID and a transaction ID to the generated data. (Step S326). In this example, “03” is assigned to the message as a sequence ID. The event notification unit 62 transmits the message to which the ID is assigned to the display device 4 via the WebSocket session (step S327).

  The control unit 141 receives a message from the information processing apparatus 10. Based on the transaction ID and sequence ID of the received message, the control unit 141 identifies the order of processing corresponding to the received message (step S328). In this example, although the received message is given “03” as the sequence ID, the message with the sequence ID “01” has not been received yet. Therefore, the control unit 141 waits for reception of a message with the sequence ID “01” (step S329).

  Now, after step S329, the message transmitted in step S315 is received by the display device 4. In this case, the control unit 141 specifies the order of processing corresponding to the received message based on the transaction ID and the sequence ID attached to the received message (step S330). Since the message with the sequence ID “01” is received, the control unit 141 executes a display process corresponding to the received message. That is, for example, the control unit 141 displays a screen indicating that the instructed process has been accepted on the display unit 144.

  Subsequently, the control unit 141 executes display processing corresponding to a message that has already been received. That is, the control unit 141 displays the display process corresponding to the message with the sequence ID “02” (the contents described in the message received in step S320 (for example, the status information C2 shown in FIG. 7B)) on the display unit 144. Display processing) is executed (step S331). Further, the control unit 141 executes a display process corresponding to the message with the sequence ID “03” (a process for displaying on the display unit 144 that the instructed copy process is completed) (step S332).

  By the way, in software that uses a plurality of communication protocols, the reception order of messages transmitted via the communication network may be changed from the transmission order. In this case, when display processing corresponding to each message is executed in the order in which the messages are received, the display may be inappropriate. For example, in the example shown in FIG. 16, when display processing corresponding to each message is executed in the order in which the messages are received, the copy processing is completed (step S <b> 332 in FIG. 16) and then copied. A display indicating that the processing request has been accepted (step S330) is performed, which may confuse the user. On the other hand, in this embodiment, it is possible to prevent the processing order from being switched by attaching a sequence ID indicating the processing order to a message transmitted via each communication path of a plurality of communication protocols.

[2] Modification The above-described embodiment is merely an example of the present invention, and may be modified as follows. Moreover, you may implement combining embodiment mentioned above and each modification shown below as needed.

[2-1] Policy of Processing Corresponding to Message In the above embodiment, when there is a message that has not been received (hereinafter referred to as “unreceived message”), the control unit 141 receives the unreceived message. Until that time, the processing after the processing order of the unreceived message is not executed and the process waits. The processing policy is not limited to this.

  For example, the control unit 141 waits until an unreceived message is received, and if the unreceived message is not received even after waiting for a predetermined period or longer, the process after the unreceived message is performed. You may perform the process corresponding to the message which is an order.

  As another example, for example, the control unit 141 may execute processing corresponding to a received message regardless of whether there is a message that has not been received. In this case, when the message is received, the control unit 141 corresponds to the received message when the processing order corresponding to the received message is earlier than the processing order of the executed process. The message may be discarded without executing the process.

  As another example, when a plurality of messages are received and the screen display cannot catch up, the control unit 141 may thin out and execute processing corresponding to the messages. For example, when a plurality of messages indicating the progress of copy processing (10%, 20%, 30,...) Are received, the control unit 141 performs thinning display without performing display corresponding to all messages. Also good. That is, when there are a plurality of received messages for which the corresponding process has not yet been executed, the control unit 141 may select a part of the plurality of messages and execute the process corresponding to the selected message. In this case, the process corresponding to the unselected message is not executed and the message is discarded. As a mode of selecting a message, for example, the control unit 141 may randomly select a message according to a predetermined random function, and the attribute satisfies a predetermined condition from a plurality of messages. May be selected. As the message attribute, for example, information indicating the priority of the message or the message type may be used. The attribute of the message may be given to the message by the middleware layer 70 of the information processing apparatus 10, the reception / response unit 61 and / or the event notification unit 62. In this case, the control unit 141 determines whether or not to execute processing corresponding to the message according to the attribute of the received message.

  In addition, the control unit 141 may instruct which policy to use according to the type of processing requested. In this case, the control unit 141 may include a parameter indicating which policy is selected in the instruction data.

[2-2] Communication Standard In the embodiment, the user device 20 includes a communication unit that performs wireless communication in conformity with the wireless LAN standard. Similarly, a communication unit that performs communication conforming to the standard of the wired LAN may be provided. In this case, the user device 20 is connected to a wired LAN router or hub via a communication cable. In this case, the communication unit of the user device 20 also functions as a communication unit that communicates with the above-described other communication devices (such as a router and a hub).

  In the embodiment, the first communication unit of the information processing device 10 and the communication unit of the operation device 30 communicate according to the standard of the wired LAN. However, the present invention is not limited to this. For example, a USB (Universal Serial Bus) And communication conforming to standards such as HDMI (registered trademark) (High-Definition Multimedia Interface), and other standards such as wireless LAN, Bluetooth (registered trademark), NFC (Near field communication), etc. Compliant wireless communication may be performed. In any case, the communication unit of the controller device 30 communicates with the first communication unit of the information processing apparatus 10 without the other routers passing through a communication device such as a hub. As described above, the communication unit of the operation device 30 functions as a communication unit that communicates with the information processing device 10 without using other communication devices.

  In the above-described embodiment, the information processing apparatus 10 and the display apparatus 4 exchange data using two types of communication protocols, HTTP and WebSocket, but the communication protocol used is not limited to these. . The communication protocol used by the information processing apparatus 10 and the display apparatus 4 includes a first communication protocol for the information processing apparatus 10 (web server) to transmit a response to a request transmitted from the display apparatus 4 (web client), and information. In order to transmit a notification from the processing device 10 to the display device 4, a second communication protocol may be included.

  Further, in the above-described embodiment, the information processing apparatus 10 and the display apparatus 4 may exchange data that conforms to the SOAP (Simple Object Access Protocol) standard. Further, the protocol that is used when the information processing device 10 and the display device 4 exchange data is not limited to this. The information processing device 10 and the display device 4 may communicate in accordance with a standard such as REST (Representational State Transfer) or SGML (Standard Generalized Markup Language).

[2-3] Configuration of Operating Device The operating device 30 includes a control unit, a storage unit, and a communication unit as illustrated in FIG. 5 in the embodiment, but may not include these.
FIG. 17 is a diagram illustrating an example of a hardware configuration of the information processing apparatus 10a according to the present modification. In the example of FIG. 17, the information processing apparatus 10 a including the units excluding the operation device 30 among the units illustrated in FIG. 6 and the operation device 30 a are illustrated. Also in this modification, the operating device 30a is fixed to the housing of the information processing device 10a. The operating device 30a includes a communication unit 33 and a UI unit 36 illustrated in FIG.

  The communication unit 33 and the UI unit 36 are connected to a bus in the information processing apparatus 10 a and are controlled by the control unit 11. The communication unit 33 is also connected to the first communication unit 15. The UI unit 36 includes a display surface and a touch sensor superimposed on the display surface, and receives an operation from the user and displays an image. The UI unit 36 supplies operation data corresponding to the received user operation to the control unit 11, and the control unit 11 performs processing according to the operation data. In this modification, the function of the display device 4 shown in FIG. 7 is realized by the control unit 11 executing a browser program.

[2-4] Installation of operating device The operating device is fixed to the housing of the information processing device in the embodiment and the above-described modified examples, but may not be fixed. For example, the controller device may be a device used as a so-called remote controller without being fixed to the housing of the information processing device.
FIG. 18 is a diagram illustrating an example of a hardware configuration of the information processing apparatus 10b according to the present modification. In the example of FIG. 18, an information processing device 10 b including each unit other than the operation device 30 among the units illustrated in FIG. 6 and the operation device 30 b are illustrated. The operating device 30b is a device that includes the units illustrated in FIG. 5 and functions as the display device 4 illustrated in FIG. The casing of the information processing apparatus 10b has a portion (hereinafter referred to as “installation section”) formed in a shape in which the operation apparatus 30b can be installed.

  The operating device 30b is connected to the first communication unit 15 via a communication cable, and is used by the user while being installed in the installation unit. The operating device 30b is used as the local display device 4 that communicates with the information processing device 10 without using other communication devices, like the operating device 30 of the embodiment. The user may remove the cable from the operation device 30b, take the operation device 30b to another location, and use it for another purpose. In the example of FIG. 18, instead of the operation device 30b, another device functioning as an operation unit (for example, the user device 20 shown in FIG. 1 or the like) may be connected. Further, the controller device 30b may be connected to the communication device 3 to communicate with the information processing device 10b via the communication device 3 and the communication line 2.

[2-5] Process Performed by Information Processing Apparatus 10 In the above-described embodiment, the operation when performing the copy process has been described. However, the process performed by the information processing apparatus 10 is not limited to the copy process. The process performed by the information processing apparatus 10 may be, for example, the image reading process described above or an image forming process.

[2-6] Operation Accepting Method In the above-described embodiment, the control unit 21 accepts a user operation with a touch sensor, but the method of accepting a user operation is not limited thereto. For example, if the display device 4 is a tablet terminal, an operation may be received using a button provided on the housing. If the display device 4 is a personal computer, an operation may be received using a keyboard, a mouse, or the like.

[2-7] Transmission of Message In the embodiment, the event notification unit 62 transmits notification data compliant with the WebSocket standard, but the present invention is not limited to this. For example, the event notification unit 62 maintains the HTTP request transmitted from the UI control unit 41 of the display device 4 using a technique called Comet, and a message (update of status information, etc.) to be transmitted occurs. In addition, the event notification unit 62 may transmit the status information in response to the maintained HTTP request. In the method using Comet, the processing status is transmitted to the user earlier than in the case where this transmission is performed in response to a request from the operation device, as in the embodiment. become.

[2-8] Category of Invention In addition to an information processing device, a display device, a communication device, and an information processing system including these devices, the present invention provides an information processing method for realizing processing performed by these devices. It can be seen as. The present invention can also be understood as a program for causing a computer called an information processing apparatus and a display apparatus to function as the above-described units. This program is provided in the form of a recording medium such as an optical disk storing the program, or in the form of being downloaded and installed on a computer via a communication line such as the Internet, etc. It may be a thing to do.

DESCRIPTION OF SYMBOLS 1 ... Information processing system, 2 ... Communication line, 3 ... Communication apparatus, 4 ... Display apparatus, 10 ... Information processing apparatus, 20 ... User apparatus, 30 ... Operation apparatus, 11, 21, 31 ... Control part, 12, 22, 32 ... Storage unit, 13 ... Image reading unit, 14 ... Image forming unit, 15 ... First communication unit, 16 ... Second communication unit, 17 ... Connection unit, 23, 33 ... Communication unit, 24, 34 ... Display unit, 25, 35 ... operation unit, 101 ... first establishment means, 102 ... second establishment means, 103 ... first reception means, 104 ... first processing execution means, 105 ... granting means, 106 ... first transmission means, 107 ... Second transmission means 401 ... third establishment means 402 ... fourth establishment means 403 ... third transmission means 404 ... second reception means 405 ... third reception means 406 ... identification means 407 ... second processing Execution means.

Claims (9)

Software functioning as a client; first establishing means for establishing a first communication path using a request / response type first communication protocol;
A second establishing means for establishing a second communication path by a second communication protocol different from the software and the first communication protocol;
Receiving means for receiving a processing execution request from the software via the first communication path;
Processing execution means for executing the processing according to the received execution request;
The first message and a second message indicating the processing status of the processing indicating that accepts the execution request, the transaction number for identifying the process, as well, each in the message group including the first message and the second message An assigning means for assigning a sequence number indicating the processing order of messages;
First transmission means for transmitting the first message assigned with the transaction number and the sequence number to the software through the first communication path without waiting for completion of the processing ;
An information processing apparatus comprising: a second transmission unit configured to transmit the second message assigned with the transaction number and the sequence number to the software through the second communication path during execution of the process and when the process is completed. . Software functioning as a server; first establishing means for establishing a first communication path by a request / response type first communication protocol;
A second establishing means for establishing a second communication path by a second communication protocol different from the software and the first communication protocol;
First transmission means for transmitting a process execution request to the software via the first communication path;
First receiving means for receiving a first message indicating that the execution request has been received from the software via the first communication path;
Second receiving means for receiving, from the software, a second message indicating a processing status during execution of the processing and completion of the processing via the second communication path;
A transaction number for identifying a process corresponding to the transmitted execution request, attached to each of the received first message and the received second message, and the first message and the second message. A specifying means for specifying the processing order of the first message and the second message based on a sequence number indicating the processing order of each message in the message group including:
An information processing apparatus comprising: a process execution unit that executes processes corresponding to each of the received first message and the received second message in the specified processing order. The information processing apparatus according to claim 2, wherein the processing execution unit does not execute processing corresponding to the first message and the second message in a processing order after the first message or the second message that has not been received from the software. . If the first message or the second message that has not been received has not been received even after waiting for a predetermined period or longer, the process execution means is subsequent to the first message or the second message that has not been received. The information processing apparatus according to claim 3, wherein processing corresponding to a first message and a second message that are processing orders is executed. When the processing order corresponding to the received first message or the second message is earlier than the processing order of the processing that has been executed, the processing execution means is configured to receive the received first message or second message. The information processing apparatus according to claim 2, wherein processing corresponding to two messages is not executed. When there are a plurality of received first messages and second messages for which the corresponding processing has not yet been executed, the processing execution means has at least one of the plurality of first messages and second messages. The information processing apparatus according to any one of claims 2 to 5, wherein a process corresponding to at least one of a first message and a second message satisfying a predetermined condition for an attribute is executed. A first information processing device;
A second information processing device,
The first information processing apparatus
A first software functioning as a client; a first establishing means for establishing a first communication path by a request / response type first communication protocol;
A second establishing means for establishing a second communication path by a second communication protocol different from the first software and the first communication protocol;
First receiving means for receiving a process execution request from the first software via the first communication path;
First process execution means for executing the process according to the received execution request;
The first message and a second message indicating the processing status of the processing indicating that accepts the execution request, the transaction number for identifying the process, as well, each in the message group including the first message and the second message An assigning means for assigning a sequence number indicating the processing order of messages;
First transmission means for transmitting the first message assigned with the transaction number and the sequence number to the first software through the first communication path without waiting for completion of the processing ;
A second transmission unit configured to transmit the second message to which the transaction number and the sequence number are assigned to the first software through the second communication path during execution of the process and when the process is completed ;
The second information processing apparatus
A second software functioning as a server, and a third establishing means for establishing the first communication path by the first communication protocol,
It said second software, and a fourth establishing means for establishing the second communication path according to the first communication protocol is different from said second communication protocol,
Third transmission means for transmitting a process execution request to the second software via the first communication path;
Second receiving means for receiving a first message indicating that the execution request has been accepted from the second software via the first communication path;
Third receiving means for receiving a second message indicating the processing status and the completion of the processing during execution of the processing corresponding to the execution request transmitted from the second software via the second communication path;
A transaction number for identifying a process corresponding to the transmitted execution request, attached to each of the received first message and the received second message, and the first message and the second message. A specifying means for specifying the processing order of the first message and the second message based on a sequence number indicating the processing order of each message in the message group including:
And a second process execution unit configured to execute processes corresponding to each of the received first message and the received second message in the specified process order. On the computer,
A software functioning as a client; and establishing a first communication path using a request / response type first communication protocol;
Establishing a second communication path with the software and a second communication protocol different from the first communication protocol;
Receiving an execution request for processing from the software via the first communication path;
Executing the process in accordance with the received execution request;
The first message and a second message indicating the processing status of the processing indicating that accepts the execution request, the transaction number for identifying the process, as well, each in the message group including the first message and the second message Assigning a sequence number indicating the message processing order;
Transmitting the first message with the transaction number and the sequence number to the software via the first communication path without waiting for the completion of the processing ;
A program for executing the step of transmitting the second message to which the transaction number and the sequence number are assigned to the software through the second communication path during execution of the process and when the process is completed . On the computer,
A software functioning as a server; and establishing a first communication path using a request / response type first communication protocol;
Establishing a second communication path with the software and a second communication protocol different from the first communication protocol;
Transmitting a process execution request to the software via the first communication path;
Receiving a first message indicating that the execution request has been accepted from the software via the first communication path;
Receiving from the software a second message indicating a processing status during execution of the processing and completion of the processing via the second communication path;
A transaction number for identifying a process corresponding to the transmitted execution request, attached to each of the received first message and the received second message, and the first message and the second message. Identifying the processing order of the first message and the second message based on a sequence number indicating the processing order of each message in the message group including:
Executing a process corresponding to each of the received first message and the received second message in the specified processing order.

Images