JP6418290B2 - Information processing apparatus, data structure of image file and program - Google Patents

Description

  The present invention relates to an information processing apparatus, a data structure of an image file, and a program.

  Patent Document 1 describes a format for describing a tag defined using a markup language in attribute information of an image file, that is, an Exif (Exchangeable image file) format.

Japanese Patent Laid-Open No. 2004-120069

  In the conventional JPEG (Joint Photographic Experts Group) format, simple attribute information such as a shooting date is allowed to be described. However, it is not assumed that information defining workflow processing related to the subject is described in the execution language in the attribute information.

  An object of the present invention is to improve the convenience of an image compared to a case where information defining workflow processing related to a subject is not described in part of attribute information attached to an image data file. .

According to the first aspect of the present invention, a recognition unit that recognizes a user's designation for an image including each subject to which attribute information is attached, and the attribute information that is attached to the subject designated by the user on the image, And a control unit that requests notification of information related to the subject specified by the user through execution of a workflow process specified by the read attribute information after reading from the attribute information attached to the data file. Information processing apparatus.
The invention according to claim 2 is the information processing apparatus according to claim 1, wherein the data file of the image conforms to a JPEG format, and the attribute information is described in JSON.
According to a third aspect of the present invention, in the information processing apparatus according to the first aspect, the control unit displays an operation screen for operating the device as the subject through the execution of the workflow process. It is.
According to a fourth aspect of the present invention, the control unit transmits or transmits a command related to an operation received through the operation screen to the device as the subject. Device.
The invention according to claim 5 is characterized in that the control unit changes the display of the subject in the image so that the state realized by the operation is reflected. Information processing apparatus.
The invention according to claim 6 is characterized in that the control unit displays any one or more of the plurality of operation screens prepared according to the device as the subject. Information processing apparatus.
The invention according to claim 7 is the information processing apparatus according to claim 3, wherein the display of the subject is changed so as to reflect the state of an actual device corresponding to the subject.
The invention according to claim 8 is the information processing apparatus according to claim 1, wherein the control unit reproduces an audio file associated with the subject through execution of the workflow process.
According to the ninth aspect of the present invention, in the workflow process, when the subject is an image of a device that can be operated by a user, an operation screen for operating the device is displayed, and the subject cannot be operated by the user. 2. The information processing apparatus according to claim 1, wherein the content of the process is switched so as to notify information about the subject.
According to a tenth aspect of the present invention, the control unit is described in the document of the document file to which the image is pasted and the contents of the information described in the attribute information attached to the data file of the image. The information processing apparatus according to claim 1, wherein a single function is executed in combination with the content being stored.
According to an eleventh aspect of the present invention, the control unit is registered in an application that opens the document file to which the image is pasted and the content of the information described in the attribute information included in the data file of the image. The information processing apparatus according to claim 1, wherein one function is executed in combination with the contents of the information processing apparatus.
The invention according to claim 12 is the information processing apparatus according to claim 1, wherein the control unit acquires a decoding result of the information from an external device and executes the workflow process.
The invention according to claim 13 detects an area portion corresponding to a subject specified by the user on an image including each subject to which attribute information is attached, or detects an area portion on the image where a predetermined subject exists. And a control unit that describes, in a part of the attribute information attached to the data file of the image, information for requesting notification of information related to the subject of the detected area portion through execution of a workflow process And an information processing apparatus.
According to a fourteenth aspect of the present invention, in the data structure of the image file processed by the information processing apparatus, the first data area for storing the main body of the image and the second data for storing attribute information of the main body of the image A second data area including attribute information attached to each subject included in the image as part of the attribute information of the main body, and the subject is designated by the user on the image In this case, the data structure of the image file requests the information processing apparatus to notify the information related to the subject specified by the user through the execution of the workflow process defined by the attribute information attached to the subject.
In the invention according to claim 15, the computer is configured to recognize a user's designation for an image including each subject to which attribute information is attached, and the attribute information attached to the subject designated by the user on the image. A control unit that requests reading of information related to the subject specified by the user through execution of a workflow process specified by the read attribute information after reading from the attribute information attached to the data file of the image It is a program to make it function as.
According to the sixteenth aspect of the present invention, the computer detects an area portion corresponding to the subject instructed by the user on the image including each subject to which attribute information is attached, or an area on the image where a predetermined subject exists. Information for requesting notification of information related to the subject of the detected area portion through execution of a workflow process in a part of the attribute information attached to the data file of the image It is a program for functioning as a control unit to be described.

According to the first aspect of the present invention, the convenience of the image is improved as compared with the case where information defining workflow processing related to the subject is not described in part of the attribute information attached to the image data file. To do.
According to the second aspect of the present invention, the convenience of the image conforming to the JPEG format is improved as compared with the case where the workflow processing related to the subject is not described in JSON in the attribute information of the JPEG format.
According to the third aspect of the present invention, the convenience of the image is improved as compared with the case where the operation screen of the device that is the subject is not displayed.
According to the fourth aspect of the present invention, the convenience of the image is improved as compared with the case where the command related to the operation is not transmitted to the device as the subject.
According to the fifth aspect of the present invention, it is easier to confirm the operation result for the device that is the subject than when the display of the subject of the image does not change.
According to the sixth aspect of the present invention, the convenience of the image is improved as compared with the case where a plurality of operation screens are not prepared according to the device that is the subject.
According to the seventh aspect of the present invention, it is easier to confirm the operation result for the device that is the subject than when the display of the subject of the image does not reflect the state of the actual device.
According to the invention described in claim 8, the convenience of the image is improved as compared with the case where the audio file associated with the subject is not reproduced.
According to the ninth aspect of the invention, information according to the subject included in the image can be obtained, so that the convenience of the image is improved.
According to the tenth aspect of the present invention, compared to the case where one function is not executed by combining the information described in the attribute information of the image and the content described in the document of the document file, the linked functions Can be executed.
According to the eleventh aspect of the present invention, the information described in the attribute information of the image is combined with the contents registered in the application that opens the document file to which the image is pasted, and compared with the case where one function is not executed. Thus, linked functions can be executed.
According to the twelfth aspect of the present invention, the convenience of the decoding result is improved as compared with the case where the attached information of the image is not given to the external device and is not decoded.
According to the thirteenth aspect of the present invention, compared to the case where the information for requesting the notification of information through the execution of the workflow process related to the subject is not described in a part of the attribute information attached to the image data file, the image Improved convenience.
According to the fourteenth aspect of the present invention, the information for requesting notification of information through the execution of the workflow process related to the subject is compared with the data structure of the image file that does not include a part of the attribute information attached to the image data file. Thus, the convenience of the image is improved.
According to the fifteenth aspect of the present invention, the convenience of the image is improved as compared with the case where information defining workflow processing related to the subject is not described in part of the attribute information attached to the image data file. To do.
According to the sixteenth aspect of the present invention, as compared with the case where information for requesting notification of information through execution of a workflow process related to a subject is not described in a part of attribute information attached to the image data file, the image Improved convenience.

It is a figure which shows the example of a data structure of the JPEG file used by this Embodiment. It is a figure which shows the structural example of the image processing system used by this Embodiment. It is a figure which shows the structural example of the computer which concerns on this Embodiment. 1 is a diagram illustrating a configuration example of an image forming apparatus according to an exemplary embodiment. It is a figure which shows an example of the still image used with each utilization form. It is a block diagram which shows an example of a function structure of the control part expressed from the viewpoint of the function which processes the JPEG file which contains the information which prescribes | regulates a workflow process in attribute information. It is a flowchart which shows an example of the procedure of the process performed by a control part. It is a figure explaining the output of a command among the usage modes 1. FIG. It is a figure explaining the example of a change of the display mode added to a still image among the usage forms 1. FIG. It is a figure explaining the operation example in the case of copying the JPEG file in which the information which prescribes | regulates a workflow process is described in attribute information. It is a figure explaining the other operation example in the case of copying the JPEG file in which the information which prescribes | regulates a workflow process is described in attribute information. It is a figure which shows the example of a pop-up screen which confirms the duplication of the information which prescribes | regulates the workflow process displayed at the timing of a JPEG file duplication. It is a figure explaining the operation example in the case of deleting the subject in which the information which prescribes | regulates a workflow process is described in attribute information from the corresponding still image by image editing. It is a figure explaining the operation example in case one subject in which the information which prescribes | regulates a workflow process is described by attribute information is duplicated or cut out by image editing. It is a figure which shows the example which has arrange | positioned the small image of the electronic device copied or cut out from several still images on one still image. It is a figure explaining the example of a screen which appears when the image of the JPEG file in which the information which prescribes | regulates a workflow process is described in attribute information is pasted on an electronic document. FIG. 10 is a diagram illustrating another example of a screen that appears when an image of a JPEG file in which information defining workflow processing is described in attribute information is pasted on an electronic document. It is a figure explaining the utilization form which embeds and prints the low visibility coded image showing the content of attribute information in a still image. It is a flowchart which shows an example of the process performed by a control part when printing a JPEG file. It is a figure explaining a mode that a coded picture and a still picture are separated from a synthetic picture, and attribute information is restored from a coded picture. It is a flowchart which shows an example of the process performed by a control part, when the coded image produced | generated from the attribute information is embedded in the printed image. It is a figure explaining the case where the information which prescribes | regulates a workflow process is described in relation to a person. It is a block diagram which shows an example of a function structure of the control part expressed from the viewpoint which records the information which prescribes | regulates a workflow process. It is a figure explaining the example of designation | designated of the image area by the user. It is a figure explaining the writing to the attribute information of the information which prescribes | regulates a workflow process.

  Embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In the following, an embodiment in which the present invention is applied to a still image file will be described, but it may be applied to a moving image file. In the embodiment described later, an example of a JPEG file conforming to the JPEG format will be described for the sake of convenience. However, the present invention may be applied to other still image files including attribute information as part of data.

<Data structure of still image file>
FIG. 1 is a diagram showing the data structure of a JPEG file 10 used in the present embodiment. The JPEG file 10 is an example of an image data file and conforms to the JPEG format.

  The JPEG file 10 includes an image start segment (Start Of Image segment: SOI) 11 indicating the start position of an image, a type 1 application segment (Application type 1 segment: App1) 12 used for description of Exif information, and the like. A type 11 application segment (Application type 11 segment: App11) 13, an image data (Image Data: ID) 14, and an image end segment indicating the end position of the image. (End Of Image segment: EOI) 15. Here, the image data 14 is an example of a first data area, and the type 11 application segment 13 is an example of a second data area. The image data 14 stores the main body of a still image.

  A region sandwiched between the image start segment 11 and the image end segment 15 is also called a frame. Although not shown in FIG. 1, the JPEG file also includes two segments (not shown), that is, a quantization table definition segment (Define Quantization Table segment: DQT) and a Huffman table definition segment (Define Huffman Table segment: DHT). include. Segments other than these are arranged as necessary. In the case of FIG. 1, the type 1 application segment 12 and the type 11 application segment 13 are the attribute information 16 of the JPEG file 10. Therefore, each of the type 1 application segment 12 and the type 11 application segment 13 is a part of the attribute information 16.

  In the application segment 13 of FIG. 1, information 13A and 13B that define workflow processing related to the subject included in the still image in which the JPEG file 10 is created are described. For example, the information 13A is information corresponding to the workflow process 1 related to the subject 1, and the information 13B is information corresponding to the workflow process 2 related to the subject 2. The number of pieces of information stored in the type 11 application segment 13 may be zero, one, or three or more.

  Information 13A and 13B may be associated with one subject. That is, a plurality of pieces of information may be associated with one subject. For example, the information 13A is used for output in a first language (eg, Japanese, for the first OS), and the information 13B is used for output in a second language (eg, English, for the second OS). The language in which the workflow process is output may be specified by the user through the selection screen, for example. The workflow process includes, for example, storage, display, aggregation, transmission, and acquisition of information included in the subject associated with the information 13A and 13B. In addition, the workflow processing includes display of an operation panel for controlling the operation of a real device corresponding to the subject associated with the information 13A and 13B. Information 13A and 13B may be provided for each type of operation of one device. For example, the information 13A may be for operating a channel of the television receiver, and the information 13B may be for operating a power button of the television receiver.

  Information 13A and 13B is described as text. In the present embodiment, JSON (Java Script Object Notation), which is an example of an execution language, is used for text description. JSON (registered trademark) is a language that uses a part of notation of objects in JavaScript (registered trademark) as a base of syntax. Of course, the execution language used to describe the workflow process is not limited to JSON.

<Configuration of image processing system and information processing apparatus>
FIG. 2 is a diagram illustrating a configuration example of the image processing system 100 used in the present embodiment. The image processing system 100 includes a portable computer 200 that a user uses for visual recognition of an image, and an image forming apparatus 300 that is used for printing a still image or transmitting a fax. The computer 200 and the image forming apparatus 300 here are both examples of an information processing apparatus. FIG. 2 illustrates a state in which the computer 200 and the image forming apparatus 300 are connected via a communication unit (not shown) and send and receive the JPEG file 10. However, the computer 200 and the image forming apparatus 300 may be used alone.

  As the computer 200, for example, a notebook computer, a tablet computer, a smartphone, a mobile phone, a camera, a portable game machine, or the like is used. The image forming apparatus 300 according to the present embodiment is an apparatus equipped with a copy function, a scanner function, a fax transmission / reception function, and a printing function. However, the image forming apparatus 300 may be an apparatus specialized for one function, such as a scanner, a fax machine, a printer (including a three-dimensional printer), or an image editing apparatus.

  FIG. 3 is a diagram illustrating a configuration example of the computer 200 according to the present embodiment. The computer 200 includes a control unit 210 that controls the entire apparatus, a storage unit 214 that is used to store the JPEG file 10 and the like, a display unit 215 that is used to display an image, and an operation reception unit 216 that receives user input operations. And a communication unit 217 used for communication with an external apparatus (for example, the image forming apparatus 300). Each unit described above is connected to the bus 218 and exchanges data via the bus 218.

  The control unit 210 is an example of a control unit, and includes a CPU (Central Processing Unit) 211, a ROM (Read Only Memory) 212, and a RAM (Random Access Memory) 213. The ROM 212 stores a program executed by the CPU 211. The CPU 211 reads a program stored in the ROM 212 and executes the program using the RAM 213 as a work area. Through the execution of the program, the workflow process defined by the information 13A and 13B described above is executed. A specific example of the workflow process will be described later.

  The storage unit 214 includes a storage device such as a hard disk device or a semiconductor memory. The display unit 215 is a display device that displays various images through execution of a program (including an operation system and firmware). The display unit 215 is configured by, for example, a liquid crystal display panel or an organic EL (Electro Luminescence) display panel. The operation reception unit 216 is a device that receives an operation from a user, and includes, for example, a keyboard, a button, a switch, a touch pad, a touch panel, and the like. The communication unit 217 is configured by, for example, a LAN (Local Area Network) interface.

  FIG. 4 is a diagram illustrating a configuration example of the image forming apparatus 300 according to the present embodiment. The image forming apparatus 300 includes a control unit 310 that controls the entire apparatus, a storage unit 314 that is used to store the JPEG file 10 and the like, a display unit 315 that is used to display an operation reception screen and a still image, and user input An operation receiving unit 316 that receives an operation, an image reading unit 317 that reads an image of a set original and generates image data, and an image that forms an image on a sheet that is an example of a recording material using, for example, an electrophotographic method or an inkjet method A communication unit 319 used for communication between the forming unit 318 and an external device (for example, the computer 200), and an image processing unit 320 that performs image processing such as color correction and gradation correction on the image represented by the image data. Each unit described above is connected to the bus 321 and exchanges data via the bus 321.

  The control unit 310 is an example of a control unit, and includes a CPU (Central Processing Unit) 311, a ROM (Read Only Memory) 312, and a RAM (Random Access Memory) 313. The ROM 312 stores a program executed by the CPU 311. The CPU 311 reads a program stored in the ROM 312 and executes the program using the RAM 313 as a work area. Each part of the image forming apparatus 300 is controlled through execution of the program. For example, image formation on the paper surface, generation of a read image, and the like are controlled.

  The storage unit 314 includes a storage device such as a hard disk device or a semiconductor memory. The display unit 315 is a display device that displays various images through execution of a program (including an operation system and firmware). The display unit 315 includes, for example, a liquid crystal display panel or an organic EL (Electro Luminescence) display panel. The operation receiving unit 316 is a device that receives an operation from a user, and includes, for example, a button, a switch, a touch panel, and the like.

  The image reading unit 317 is a so-called scanner device. The image forming unit 318 is a print engine that forms an image on a sheet that is an example of a recording material. The communication unit 319 is configured by, for example, a LAN (Local Area Network) interface. The image processing unit 320 is configured by a dedicated processor that performs image processing such as color correction and gradation correction on image data, for example.

<Example of still image>
First, an example of a still image used in each usage mode will be described. Since the moving image is configured as a time sequence of a plurality of still images, the still image described below is also applied to a moving image. FIG. 5 is a diagram illustrating an example of a still image used in each usage mode. The still image 400 displayed on the display unit 215 corresponds to an electronic photograph stored in a recording medium when the inside of an office is photographed with a digital camera, for example. As described above, the still image 400 is stored in the image data 14 of the JPEG file 10. The still image 400 includes an image forming apparatus 401, a television receiver 402, a lighting fixture 403, a person 404, and a planting plant 405 as subjects. In the case of the present embodiment, the attribute information 16 of the JPEG file 10 corresponding to the still image 400 describes information 13A associated with at least one of these five subjects.

<Configuration related to decryption function>
Each usage form to be described later is realized by the computer 200, the image forming apparatus 300, or the cooperation of the computer 200 and the image forming apparatus 300. In the following description, each usage mode is realized by the computer 200 unless otherwise specified. Unless otherwise specified, it is assumed that the attribute information 16 of the JPEG file 10 describes one piece of information 13A for one subject. The information 13A is information that defines workflow processing related to one subject, and is described in JSON.

  FIG. 6 is a block diagram showing an example of the functional configuration of the control unit 210 expressed from the viewpoint of the function of processing the JPEG file 10 that includes information 13A defining workflow processing in the attribute information 16. The control unit 210 includes an instruction recognition unit 221 used for recognizing a user instruction input through the operation reception unit 216, and an execution control unit that controls execution of information 13A that defines workflow processing related to the subject. It functions as 222. Here, the instruction recognition unit 221 is an example of a recognition unit, and the execution control unit 222 is an example of a control unit.

  The user's instruction is recognized as selection of a subject included in the still image 400. The user's designated position is given as coordinates (pixel value, pixel value) with respect to a coordinate system defined for the still image 400 (for example, a coordinate system with the upper left corner of the screen as the origin). The indicated position may be recognized as the position of the cursor displayed superimposed on the still image 400, and the touch panel sensor arranged on the near side (user side) with respect to the display unit 215 is recognized as the position touched by the user. Also good.

  The execution control unit 222 executes the following process when the information 13A defining the workflow process related to the subject is described in a part of the attribute information 16 attached to the JPEG file 10. First, the execution control unit 222 determines whether or not the instruction position recognized by the instruction recognition unit 221 is included in the region or range associated with the information 13A. If the indicated position recognized by the instruction recognizing unit 221 is not included in the area or range associated with the information 13A, the execution control unit 222 does not execute the work process defined by the information 13A. On the other hand, when the instruction position recognized by the instruction recognition unit 221 is included in an area or range associated with the information 13A, the execution control unit 222 executes a workflow process defined by the information 13A.

  Next, a procedure of processing executed by the control unit 210 will be described. FIG. 7 is a flowchart illustrating an example of a procedure of processing executed by the control unit 210. First, when the control unit 210 reads the JPEG file 10 corresponding to the still image 400 displayed on the display unit 215, the control unit 210 reads the attribute information 16 attached to the JPEG file 10 (step 101).

  Next, the control unit 210 recognizes the position of the mouse pointer on the still image 400 displayed on the display unit 215 (step 102). Thereafter, the control unit 210 determines whether or not the information 13A described in JSON is associated with the position designated by the mouse pointer (step 103). If a negative result is obtained in step 103, the control unit 210 returns to step 102. This means that the information 13A is not associated with the area on the still image 400 indicated by the mouse pointer. On the other hand, when a positive result is obtained in step 103, the control unit 210 executes a workflow process described in JSON (step 104). The contents of the workflow process to be executed differ depending on the contents to be described.

<Usage form>
Hereinafter, a usage form realized through execution of the information 13A described in the application segment 13 of the attribute information 16 will be described.

<Usage form 1>
Here, a case will be described in which the attribute information 16 of the JPEG file 10 describes information 13A that defines workflow processing related to the lighting fixture 403 that is one of the subjects of the still image 400. That is, in the case of the usage mode 1, the information 13A corresponding to the image forming apparatus 401, the television receiver 402, the person 404, and the planting plant 405 is not described in the attribute information 16.

  The workflow processing in this usage mode 1 corresponds to confirmation of the user's designated position, display of an operation screen for controlling lighting and extinction of the lighting fixture 403, reception of operation input for the displayed operation screen, and accepted operation input The command signal is output and the display mode of the lighting fixture 403 is changed in this order.

  Hereinafter, the state of execution of the workflow process in the usage mode 1 will be described with reference to FIGS. 8 and 9. FIG. 8 is a diagram illustrating command output up to usage mode 1. FIG. 9 is a diagram for explaining a change example added to the still image 400 in the usage pattern 1.

  First, the user displays the still image 400 on the screen of the display unit 215. Then, the attribute information 16 of the still image 400 is given to the execution control unit 222. The execution control unit 222 decrypts the contents described in the attribute information 16 and specifies an area or range associated with the information 13A described in the application segment 13.

  Next, the user moves the mouse pointer 501 over the lighting fixture 403 in the still image 400 (step indicated by a circled number 1 in the figure). When the touch panel sensor is arranged on the front side of the display unit 215, this operation is performed by a touch operation with a fingertip. Note that the lighting fixture 403 in the still image 400 is in a lighting state because it is in a state at the time of shooting. This user operation input is received through the operation receiving unit 216 and given to the instruction recognizing unit 221.

  In the case of this usage mode, since the information 13A is associated with the lighting fixture 403, the execution control unit 222 executes the workflow process described in the information 13A. First, a pop-up screen 510 for operating the lighting fixture 403 is displayed on the screen of the display unit 215 (step indicated by a circled number 2 in the figure). On the pop-up screen 510, an on button 511 and an off button 512 are shown. Next, the user moves the mouse pointer 501 on the off button 512 and clicks the off button 512. This operation input is given from the operation reception unit 216 to the instruction recognition unit 221. The pop-up screen 510 is an example of a screen associated with the subject.

  When the execution control unit 222 recognizes that the off button 512 has been clicked, the execution control unit 222 transmits a turn-off command to the actual lighting fixture 601 shown in the still image 400 (step indicated by a circled number 3 in the figure). ). Command signals necessary for controlling the lighting apparatus 601 are registered in the computer 200 in advance. In addition, when the lighting fixture 601 has an infrared light receiving unit and the turning on and off thereof is executed by receiving an infrared signal, the execution control unit 222 uses an infrared light emitting unit (not shown) mounted on the computer 200. Outputs the turn-off command.

  As a result, the actual lighting fixture 601 is switched from the lit state to the unlit state. That is, the still image 400 is used as a controller for the actual lighting fixture 601. The output destination of the turn-off command may be a remote controller that actually exists for the operation of the lighting fixture 601. In this case, a turn-off command is transmitted to the lighting fixture 601 via the remote controller.

  Incidentally, since the JPEG file 10 corresponding to the still image 400 is digital data, it can be easily distributed to a plurality of users. In other words, it is easy for a plurality of people to share a virtual controller. Therefore, there is no restriction as in the case where a physical controller is shared by a plurality of people. For this reason, lighting and extinguishing of the actual lighting fixture 601 are operated through each computer 200. Furthermore, the actual lighting fixture 601 has a one-to-one correspondence with the lighting fixture 403 of the still image 400 that is the captured image. For this reason, intuitive specification of the control target by the user is realized. Further, as a device for making it easy to understand the control state of the subject by a plurality of users, the current operator name or the like may be displayed on a virtual controller displayed on the still image 400.

  If the existing lighting fixture 601 or its remote controller is compatible with IoT (Internet of Things), the location of the user viewing the still image 400 and the location where the actual lighting fixture 601 is attached are physically located. You may be away. However, an additional mechanism for specifying the lighting fixture 601 to be controlled is required. For specifying the lighting fixture 601, for example, information on an imaging position, unique information assigned to each device, a communication address assigned to each device, and the like are used as necessary.

  Thereafter, as shown in FIG. 9, the execution control unit 222 in the present usage mode 1 changes the display mode of the lighting fixture 403 included in the still image 400 by image processing (indicated by a circled numeral 4 in the figure). Step). For example, the display brightness of the corresponding area is reduced so as to reflect the turn-off of the lighting fixture 403. Note that an image image of the lighting fixture 403 that is turned off may be created, and the display of the lighting fixture 403 may be replaced with the created image image.

  That is, the still image 400 is used for confirming that the existing lighting fixture 601 has changed to the extinguished state. The function of changing the display mode of the subject according to the control content in this way improves the convenience for the user when the lighting device 601 is controlled to be turned on or off from a remote location. Of course, when a plurality of users control a subject reflected in the still image 400, a display form changed by controlling each subject may be reflected in each still image. Even if the still image 400 itself is different, if the same subject is shown, the situation of the same subject is acquired via the network, and the display form of the same subject shown in each still image is changed. You may make it let it.

  In the above description, the case where the lighting apparatus 403 is specified on the still image 400 has been described. However, for example, when the television receiver 402 is specified on the still image 400, a JPEG file corresponding to the still image 400 is used. Operation screens such as a power switch, a channel change button, a channel selection button, and a volume adjustment button may be displayed on the still image 400 based on the information 13A described in the ten attribute information 16. When an electric window or door is designated, an opening / closing button for the window or door may be displayed. Also in these cases, the color and shape of the subject displayed on the still image 400 may be changed by reflecting the result of the operation.

  In addition, when a plurality of functions realized through workflow processing are prepared for one still image 400, when the JPEG file 10 is read, a list of prepared functions may be displayed on the still image 400. Good. However, this display may be performed when the mouse pointer 501 indicates the subject in which the information 13A is described. In addition, when only one subject in which the information 13A is registered appears in the still image 400, when the JPEG file 10 corresponding to the still image 400 is read, even if there is no instruction from the mouse pointer 501, it is determined in advance. The workflow processing that has been performed may be executed.

  In this usage mode, the decryption function for the application segment 13 is installed in the computer 200. However, the computer 200 not equipped with this decryption function cannot execute the workflow process defined in the information 13A. In this case, the computer 200 may search for an external device equipped with the decoding function of the application segment 13 via the communication unit, and may realize the above-described function in cooperation with the discovered external device. For example, the attribute information 16 (at least the application segment 13) may be transmitted from the computer 200 to the image forming apparatus 300 to be decoded, and the decoding result may be acquired from the image forming apparatus 300.

<Usage form 2>
Here, an example of processing executed when the corresponding still image 400 is image-edited or duplicated when the attribute information 16 of the JPEG file 10 includes information 13A defining workflow processing related to the subject will be described. . Note that the control unit 210 of the computer 200 also executes the processing in the usage mode 2.

  FIG. 10 is a diagram for explaining an operation example when the JPEG file 10 in which the information 13A defining the workflow processing is described in the attribute information 16 is copied. In FIG. 10, the entire JPEG file 10, that is, the attribute information 16 is also copied. If the JPEG file 10 thus replicated is distributed to a plurality of users, a usage mode in which a plurality of persons operate actual devices corresponding to the subject through the still image 400 as described above is realized.

  FIG. 11 is a diagram for explaining another example of operation when the JPEG file 10 in which the information 13A defining the workflow processing is described in the attribute information 16 is copied. In FIG. 11, the information 13 </ b> A is deleted from the attribute information 16 when the JPEG file 10 is copied. In this case, only the user who holds the original electronic photograph has the control right from the still image 400 for the actual device corresponding to the subject. Note that when copying the JPEG file 10, whether to copy the entire attribute information 16 or to delete the information 13 </ b> A from the attribute information 16 may be selected by the user. The selection here may be performed in advance or may be performed through an operation screen displayed at the timing of duplication.

  FIG. 12 is a diagram illustrating a display example of a pop-up screen 520 for confirming the duplication of the information 13A that defines the workflow process displayed at the duplication timing of the JPEG file 10. The pop-up screen 520 is a content for requesting confirmation that the attribute information 16 of the JPEG file 10 to be copied includes the execution format information 13A and that the execution format information 13A may be copied. And are included. In addition, when the YES button 521 is selected by the user, the control unit 210 duplicates the entire attribute information 16, and when the NO button 522 is selected by the user, the control unit 210 displays the attribute information 16 excluding the information 13A. Duplicate.

  FIG. 13 is a diagram for explaining an operation example when the subject whose information 13A defining the workflow processing is described in the attribute information 16 is deleted by image editing. In FIG. 13, the information 13 </ b> A is associated with the television receiver 402, and the image of the television receiver 402 is deleted from the still image 400. In this case, the control unit 210 deletes the information 13A associated with the television receiver 402 from the attribute information 16. This deletion avoids the inconvenience that an operation screen related to a subject that does not already exist on the still image 400 is displayed.

  FIG. 14 is a diagram for explaining an operation example in the case where one subject whose information 13A defining workflow processing is described in the attribute information 16 is duplicated or cut out by image editing. In FIG. 14, the information 13 </ b> A is associated with the lighting fixture 403. Only the information 13A corresponding to the lighting fixture 403 is duplicated in the attribute information 16 of the JPEG file 10 newly created for the image portion of the lighting fixture 403 (the portion surrounded by the frame line 530). That is, the information 13B corresponding to the television receiver 402 is not duplicated. As described above, the information 13A described in the attribute information 16 of the original still image 400 is copied to the new JPEG file 10 together with the partial image including the associated subject.

  This partial image copying function may be used to create an operation screen in which electronic devices included in the still image 400 are arranged on one screen. FIG. 15 is a diagram illustrating an example in which small images of an electronic device copied or cut out from a plurality of still images 400 are arranged on one still image 540. In the case of FIG. 15, the still image 540 includes an image of an image forming apparatus arranged in a living room, an image of a television receiver, an image of a lighting fixture, an image of an air conditioner, an image of a fan, an image of a video deck, and an entrance. It includes an image of the lighting fixtures arranged and an image of the air conditioner arranged in the children's room. As described above, the JPEG file 10 corresponding to these images includes information 13A relating to the electronic device that is the subject. Therefore, the still image 540 is used as an operation screen for a plurality of electronic devices.

<Usage form 3>
Here, provision of a new usage form realized by a combination of the JPEG file 10 and another document will be described. FIG. 16 is a diagram illustrating an example of a screen that appears when an image of the JPEG file 10 in which information 13A defining workflow processing is described in the attribute information 16 is pasted on the electronic document 550. The electronic document 550 is an example of a document file. In the case of FIG. 16, the electronic document 550 includes an area 551 in which a description in an execution language is embedded. In the area 551, for example, the contents defining the arrangement position and size of the pop-up screen 552 opened when the JPEG file 10 including the information 13A described in the execution language is arranged are described in the html language.

  In this case, the contents displayed on the pop-up screen 552 are defined by the information 13A on the JPEG file 10 side, and the arrangement position and size of the pop-up screen 552 are defined by the contents described in the area 551 on the electronic document 550 side. Thereby, complicated workflow processing that cannot be obtained only by the workflow processing defined by the information 13A is realized. It should be noted that special characters and figures may emerge depending on the combination of the information 13A and the description on the area 551 side.

  FIG. 17 is a diagram for explaining another example of a screen that appears when an image of the JPEG file 10 in which information 13A defining workflow processing is described in the attribute information 16 is pasted on the electronic document 550. FIG. 17 illustrates an example in which information 13A described in an execution language is operated in combination with the macro 601 of the application 600 that displays the electronic document 550, and the execution result is displayed as a pop-up screen 553. For example, the price information of the subject collected by the workflow process of the information 13A may be totaled using a macro. In addition, when the subject is a receipt, the information 13A may be contents that extract a fee portion and give it to a macro.

<Usage form 4>
Here, an operation in the case where an image of the JPEG file 10 in which the information 13A defining the workflow process is described in the attribute information 16 is printed on a sheet of recording material will be described. In the above-described usage mode, the attribute information 16 is copied in the form of a data file, but in this usage mode, it is performed using paper. FIG. 18 is a diagram illustrating a usage mode in which a low-visibility coded image 560 representing the content of the attribute information 16 is embedded in a still image 400 and printed.

  The low visibility coded image 560 is an image composed of inconspicuous tiny dots arranged in the background of the output document. For example, MISTCODE (Micro-dot Iterated and Superimposed Tag CODE) is used as a technique for creating the coded image 560. MISTCODE is composed of a pattern in which dots are arranged according to a certain rule, and this pattern is dispersedly arranged in a sheet to embed information. The control unit 210 of the computer 200 generates a composite image 570 in which the low-visibility coded image 560 created from the attribute information 16 is embedded in the still image 400, and gives the information to the image forming apparatus 300.

  FIG. 19 is a flowchart illustrating an example of processing executed by the control unit 210 when the JPEG file 10 is printed. First, the control unit 210 that has received a print instruction acquires the attribute information 16 from the JPEG file 10 to be printed (step 201). Next, the control unit 210 generates a coded image 560 from the attribute information 16 (step 202). However, the information 13A defining the workflow process may be deleted at the time of printing.

  Thereafter, the control unit 210 synthesizes the generated coded image 560 and the still image 400 corresponding to the image main body (that is, the image data 14) to generate a synthesized image 570 (step 203). Thereafter, the control unit 210 outputs the composite image 570 to the image forming apparatus 300 (step 204). Note that the composition processing of the coded image 560 and the still image 400 may be executed in the image forming apparatus 300.

  When the composite image 570 is received, the reverse process is executed. FIG. 20 is a diagram illustrating a state in which the coded image 560 and the still image 400 are separated from the composite image 570 and the attribute information 16 is restored from the coded image 560. The process flow in FIG. 20 is opposite to the process flow in FIG.

  FIG. 21 is a flowchart illustrating an example of processing executed by the control unit 210 when a coded image 560 generated from the attribute information 16 is embedded in a printed image. FIG. 21 shows the operation when the scan image generated by the image forming apparatus 300 equipped with the scanner is taken into the computer 200 through the communication means. Of course, the image forming apparatus 300 may execute processing described later.

  First, the control unit 210 analyzes the scanned image (step 301). Next, the control unit 210 determines whether or not there is embedded information in the scanned image (step 302). If a negative result is obtained in step 302, the control unit 210 ends without executing a process described later. On the other hand, when a positive result is obtained in step 302, the control unit 210 decodes information embedded in the scanned image (step 303). Specifically, the coded image 560 is decoded. Thereafter, the control unit 210 stores the scanned image as the JPEG file 10 and describes the decoded information in the attribute information 16 (step 304). The workflow process associated with the application segment 13 is described in JSON.

  By mounting the above processing functions on the computer 200, the JPEG file 10 including the information 13A defining the workflow processing is generated from the printed matter in which the attribute information 16 of the JPEG file 10 is embedded as the coded image 560.

<Usage form 5>
In the above-described usage mode, it is assumed that the information 13A defining the workflow process is associated with the device that is the subject. However, the information 13A that defines the workflow process may be attached to the person 404, the planted plant 405, or the like. FIG. 22 is a diagram for explaining a case where the information 13A defining the workflow process is described in association with the person 404.

  In the case of FIG. 22, when the person 404 is designated by the mouse pointer 501, the control unit 210 reads the information 13A from the attribute information 16 and executes the workflow process described in the information 13A. In this example, the personal information related to the subject A is read from the database and displayed on the pop-up screen 580 by executing the workflow process. In addition, an audio file “Thank you all” is played. The reproduced sound here is an example of sound associated with the subject.

<Usage form 6>
In the above description of the usage mode, the function executed by the computer 200 that has read the information 13A when the attribute information 16 of the JPEG file 10 includes the information 13A describing the workflow processing related to the subject has been described. In this usage mode, a case where the information 13A is recorded in the attribute information 16 of the JPEG file 10 will be described.

  FIG. 23 is a block diagram illustrating an example of a functional configuration of the control unit 210 expressed from the viewpoint of recording information 13A defining workflow processing. The control unit 210 includes a position detection unit 231 that detects an image position designated by the user, a subject detection unit 232 that detects a subject that matches an image registered using an image processing technique, and an application segment of the attribute information 16 13 functions as an attribute information description unit 233 that describes the workflow process in association with the detected position.

  FIG. 24 is a diagram for explaining an example of designation of an image region by the user. In FIG. 24, an area 590 is set so as to surround the display position of the television receiver 402 by dragging the mouse pointer 501. The position detection unit 231 that inputs the coordinate information giving the area 590 as the designated position outputs the coordinate information of the finally determined area as the position information.

  The subject detection unit 232 is used when an image of a subject that records the information 13A is registered in advance. The subject detection unit 232 matches the image data 14 (that is, the still image 400) included in the JPEG file 10 with the registered image, and outputs coordinate information where a subject matching the registered image exists as position information.

  The attribute information description unit 233 records the workflow process description in the application segment 13 of the attribute information 16 in association with the position information. Here, the description of the workflow process may be edited by the user, or a description prepared in advance may be used. Further, the workflow process is described in text described in JSON.

  FIG. 25 is a diagram for explaining the writing of the information 13A defining the workflow processing to the attribute information 16. Although the information 13A is not included in the attribute information 16 of the JPEG file 10 shown in the upper part of FIG. 25, the information 13A is added to the attribute information 16 of the JPEG file 10 shown in the lower part. In this way, workflow processing may be added to the existing JPEG file 10 later.

<Other embodiments>
Although the embodiments of the present invention have been described above, the technical scope of the present invention is not limited to the scope described in the above embodiments. In the above-described embodiment, the embodiment of the present invention has been described using a JPEG file of a still image as an example of the file format of the image. It can also be applied to file formats. It is clear from the description of the scope of the claims that various modifications or improvements added to the above embodiment are also included in the technical scope of the present invention.

DESCRIPTION OF SYMBOLS 10 ... JPEG file, 13 ... Type 11 application segment, 13A, 13B ... Information defining workflow processing, 100 ... Image processing system, 200 ... Computer, 221 ... Instruction recognition unit, 222 ... Execution control unit, 231 ... Position detection 232 ... Subject detection unit 233 ... Attribute information description unit 300 ... Image forming device 400 ... Still image 501 ... Mouse pointer 510, 520, 552, 553, 580 ... Pop-up screen 530 ... Frame line 540 ... still image, 550 ... electronic document, 551 ... area in which description in execution language is embedded, 600 ... application, 601 ... macro

Claims (16)

A recognition unit for recognizing a user's designation for an image including each subject with attribute information;
After the attribute information attached to the subject designated by the user on the image is read out from the attribute information attached to the data file of the image, the user is made through the execution of the workflow process specified by the read attribute information. A control unit that requests notification of information related to the subject specified by;
An information processing apparatus. The information processing apparatus according to claim 1, wherein the image data file conforms to a JPEG format, and the attribute information is described in JSON. The information processing apparatus according to claim 1, wherein the control unit displays an operation screen for operating a device as the subject through execution of the workflow process. The information processing apparatus according to claim 3, wherein the control unit transmits or transmits a command related to an operation received through the operation screen to the device as the subject. The information processing apparatus according to claim 4, wherein the control unit changes the display of the subject in the image so that the state realized by the operation is reflected. The information processing apparatus according to claim 3, wherein the control unit displays one or more of the plurality of operation screens prepared according to a device as the subject. The information processing apparatus according to claim 3, wherein the control unit changes a display of the subject so as to reflect a state of an actual device corresponding to the subject. The information processing apparatus according to claim 1, wherein the control unit reproduces an audio file associated with the subject through execution of the workflow process. The workflow process displays an operation screen for operating the device when the subject is an image of a device operable by the user, and displays information about the subject when the subject is an image that cannot be operated by the user. Switch the content of the process to notify,
The information processing apparatus according to claim 1. The control unit combines the content of the information described in the attribute information attached to the data file of the image with the content described in the document of the document file to which the image is pasted. The information processing apparatus according to claim 1, wherein a function is executed. The control unit combines a content of the information described in the attribute information included in the data file of the image with a content registered in an application that opens a document file to which the image is pasted. The information processing apparatus according to claim 1, wherein: The information processing apparatus according to claim 1, wherein the control unit acquires a decoding result of the information from an external apparatus and executes the workflow process. Detecting a region portion corresponding to the subject designated by the user on an image including each subject to which attribute information is attached, or detecting a region portion on the image where a predetermined subject exists;
A control unit that describes, in a part of the attribute information attached to the data file of the image, information for requesting notification of information related to the subject of the detected region portion through execution of a workflow process;
An information processing apparatus. In the data structure of the image file processed by the information processing device,
A first data area for storing the body of the image;
A second data area storing attribute information of the main body of the image, and the second data area including attribute information attached to each subject included in the image as a part of the attribute information of the main body. Prepared,
When the subject is designated by the user on the image, the information processing apparatus is requested to notify the information related to the subject designated by the user through execution of a workflow process defined by the attribute information attached to the subject. The data structure of the image file. Computer
A recognition unit for recognizing a user's designation for an image including each subject with attribute information;
The attribute information attached to the subject designated by the user on the image is read out from the attribute information attached to the data file of the image, and then the workflow process specified by the read attribute information is executed. A control unit that requests notification of information related to the subject specified by the user;
Program to function as. Computer
A detection unit that detects a region portion corresponding to a subject instructed by a user on an image including each subject to which attribute information is attached, or detects a region portion on the image in which the predetermined subject exists;
A control unit that describes, in a part of the attribute information attached to the data file of the image, information for requesting notification of information related to the subject of the detected region portion through execution of a workflow process;
Program to function as.

Images