JP6175927B2 - Image processing apparatus, image processing method, program, and image processing system - Google Patents

Description

  The present invention relates to an image processing apparatus capable of displaying an arbitrary image, and more specifically, an image processing apparatus and an image processing method for displaying a display screen of another information processing apparatus and displaying a drawing image generated by a user. , A program, and an image processing system.

Conventionally, electronic blackboards, tablet terminals, personal computers, etc. on which a background image can be displayed on a display in a meeting at a company, educational institution, administrative organization, etc., and on which a user can write a drawn image such as letters, numbers, figures, etc. Is being used.
As an example of such an electronic blackboard or the like, Patent Document 1 discloses an electronic information board system in which a computer device that controls a display device displays an insertion image superimposed on a background image on the display device. In this electronic information board system, a drawing image generated by a user using a touch panel provided on the display device can be displayed on the display device by superimposing it on the background image.
In order for the user to use the content displayed on the display device, the drawn image, or the like later, it is necessary to save these data at some timing. If the power of the electronic information board system is accidentally turned off even though the user has not saved the data, there is a problem that the data cannot be taken out.
Therefore, Patent Document 1 describes an image forming apparatus that collectively saves the contents of a volatile memory to a nonvolatile storage device at the timing of transition from a standby state to a suspended state with lower power consumption. According to Patent Document 1 , even if the system is shut down from the suspended state, the data saved in the nonvolatile storage device can be used later.

However, as in Patent Document 1 , if the contents of the volatile memory are collectively stored in the nonvolatile storage device, the state of the device before the transition to the suspend state can be completely reproduced, but data that is not always necessary is included. There is a problem that it will be.
The present invention has been made in view of the above circumstances, and is necessary for later use even when the power of the apparatus is accidentally turned off by storing necessary data at a predetermined timing. An object is to provide an image processing apparatus capable of maintaining predetermined data.

In order to solve the above-described problem, the invention according to claim 1 is an image processing apparatus that displays an image on a display unit, and uses position information indicating a contact position or an approach position of an object with respect to the display unit. A drawing generation unit that generates a drawing image, a snapshot storage unit that stores a snapshot image obtained by synthesizing the display image supplied from the outside and the drawing image, and a non-volatile that retains the stored contents even when the power is shut off Storage means and control means for controlling the operation mode to switch to the power saving mode or the non-power saving mode, and the control means controls to switch the operation mode from the non-power saving mode to the power saving mode. before, the according to the state of setting information for setting the data to be included in the return data, and the drawing position information indicating the coordinate position of the drawn image snapshot And controlling at least one of the read-out snapshot image from a shot storage means so as to be stored in the nonvolatile storage means included in the return data.

  In the present invention, before switching the operation mode from the non-power saving mode to the power saving mode, at least one of the drawing position information and the snapshot image is stored in the nonvolatile storage means. In the image processing apparatus, the drawing position information and the snapshot image are one piece of information that enables the content displayed on the display unit to be reproduced. Therefore, even when the image processing apparatus is turned off by mistake, the contents displayed on the display means can be reproduced.

It is a figure which shows one Embodiment of the image processing system of this invention. It is a figure which shows the hardware constitutions and functional structure of the image processing apparatus which concerns on embodiment shown in FIG. 6 is a flowchart illustrating processing executed when the processor of the image processing apparatus performs control for shifting from a normal operation mode to a standby mode. It is a flowchart which shows the determination process of the information included in the data for a return. 14 is a flowchart illustrating processing when an image processing apparatus according to a fourth embodiment of the present invention shifts from a normal operation mode to a standby mode. It is a flowchart which shows the process at the time of transfer from standby mode to normal operation mode. It is a figure which shows another embodiment of the image processing system of this invention. FIG. 8 is a diagram illustrating a hardware configuration and a functional configuration of the image processing apparatus according to the embodiment illustrated in FIG. 7.

Hereinafter, the present invention will be described in detail with reference to embodiments shown in the drawings. However, the components, types, combinations, shapes, relative arrangements, and the like described in this embodiment are merely illustrative examples and not intended to limit the scope of the present invention only unless otherwise specified. .
[One aspect of image processing system]
FIG. 1 is a diagram showing an embodiment of an image processing system of the present invention. The image processing system 100 includes an image processing apparatus 110 and user PCs 130a and 130b. The image processing apparatus 110 and the user PCs 130a and 130b are connected by cables 124 and 126, respectively.
The image processing apparatus 110 is an apparatus that can display display images of the user PCs 130a and 130b and display a drawing image generated by the user. Specifically, various devices such as an electronic blackboard, a tablet terminal, and a personal computer can be exemplified. Further, the image processing apparatus 110 generates an event by touching the display unit 112, and transmits this event to the user PCs 130a and 130b as an event of an input device such as a mouse or a keyboard.

The user PCs 130 a and 130 b are information processing apparatuses that provide images to be displayed on the image processing apparatus 110. The user PCs 130a and 130b are provided with an interface for outputting image signals, and supply image signals for forming display images of the user PCs 130a and 130b to the image processing apparatus 110 at a predetermined rate (for example, 30 frames per second).
In the present embodiment, the user PCs 130 a and 130 b include VGA output terminals (not shown) as interfaces, and can transmit VGA signals to the image processing apparatus 110 via a cable 124 such as a VGA cable. In another embodiment, the user PCs 130a and 130b may transmit the display image by wireless communication complying with various wireless communication protocols.

In addition, the user PCs 130a and 130b can acquire images displayed on the display unit 112 by the image processing apparatus 110. The user PCs 130a and 130b have a USB port (not shown), and use a general-purpose driver such as USB Mass Storage Class for displaying images stored in the image processing apparatus 110 connected via the USB cable 126. And can be obtained.
In the embodiment shown in FIG. 1, notebook PCs are used as the user PCs 130a and 130b. However, in other embodiments, image frames such as desktop PCs, tablet PCs, PDAs, digital video cameras, and digital cameras can be supplied. A simple information processing apparatus can be employed. In the image processing system 100 illustrated in FIG. 1, two user PCs 130a and 130b are employed. However, in another embodiment, one user PC or three or more user PCs may be employed.

FIG. 2 is a diagram showing a hardware configuration and a functional configuration of the image processing apparatus 110 according to the embodiment shown in FIG. Hereinafter, the hardware configuration and functional configuration of the image processing apparatus 110 will be described with reference to FIG.
The image processing apparatus 110 includes an image input interface 232 and an image output interface 234, and is connected to the user PCs 130a and 130b via these interfaces.
The image input interface 232 is an interface that receives an image signal forming a display image of the user PCs 130a and 130b. In the present embodiment, a DVI connector configured by a DVI (Digital Visual Interface) terminal can be employed as the image input interface 232. The image input interface 232 receives a VGA signal from the user PCs 130 a and 130 b via the cable 124 such as a VGA cable, and supplies the VGA signal to the image acquisition unit 206 provided in the image processing apparatus 110.

In other embodiments, a Video Graphics Array (VGA) connector, a High-Definition Multimedia Interface (HDMI) (registered trademark) connector, a Displayport connector, or the like may be employed. In still another embodiment, the image input interface 232 may receive image signals from the user PCs 130a and 130b by wireless communication conforming to a wireless communication protocol such as Bluetooth (registered trademark) or WiFi.
The image output interface 234 is a physical interface that outputs a display image of the image processing apparatus 110 to an external apparatus such as the user PCs 130a and 130b. In the present embodiment, a USB socket can be adopted as the image output interface 234.
The image processing apparatus 110 includes a processor 200, a ROM 202, a RAM 204, an image acquisition unit 206, a coordinate detection unit 224, a contact detection device 226, and a display unit 112.

  The processor 200 is a processing operation device such as a CPU or MPU, and operates OS such as WINDOWS (registered trademark) series, UNIX (registered trademark), LINUX (registered trademark), TRON, ITRON, μITRON, and the like of these OSs. Under the management, the program of the present invention described in a programming language such as assembler, C, C ++, Java (registered trademark), JavaScript (registered trademark), PERL, RUBY, PYTHON is executed. The ROM 202 is a non-volatile memory in which a boot program such as BIOS or EFI is stored.

The RAM 204 is a main storage device such as a DRAM or an SRAM, and provides an execution space for executing the program of the present invention. The processor 200 reads out the program of the present invention from the hard disk device 250 for continuously storing software programs, various data, and the like, expands it in the RAM 204, and executes it. The program of the present invention includes an event processing unit 210, an application image generation unit 212, a layout management unit 214, a drawing generation unit 216, a synthesis unit 218, a display control unit 220, a snapshot generation unit 222, and a repository management unit, which are program modules. 228 is included.
The image acquisition unit 206 is a functional unit that acquires image signals from the user PCs 130a and 130b. When the image acquisition unit 206 receives an image signal from the user PCs 130a and 130b via the image input interface 232, the image acquisition unit 206 analyzes the image signal and forms an image frame that is a display image of the user PCs 130a and 130b formed by the image signal. Image information such as the resolution of the image and the update frequency of the image frame is derived and transmitted to the application image generation unit 212.

Further, the image acquisition unit 206 forms image frames, which are display images of the user PCs 130a and 130b, using the image signals, and overwrites and saves them in the video RAM 208, which is a storage means capable of temporarily storing image data. To do.
The application image generation unit 212 is a functional unit that generates various display windows to be displayed on the display unit 112. The display window includes a display window that displays an image frame that is a display image of the user PCs 130a and 130b, a display window that displays a drawing image generated by the user, buttons and menus for performing various settings of the image processing apparatus 110, and the like. A display window to be displayed, a display window such as a file viewer or a Web browser, and the like are included. The application image generation unit 212 draws these display windows on an image layer in which the display window is to be drawn.
The layout management unit 214 is a functional unit that draws the display images of the user PCs 130a and 130b on the display window generated by the application image generation unit 212. When the layout management unit 214 acquires the image information from the image acquisition unit 206, the layout management unit 214 acquires the image frame stored in the video RAM 208, and the application image generation unit 212 generates the size of the image frame using the image information. The image frame is changed so as to match the size of the display window, and the image frame is drawn on the image layer to be drawn.

  The contact detection device 226 is a functional unit that detects contact of an object such as the drawing device 240. In the present embodiment, as the contact detection device 226, a coordinate input / detection device using an infrared ray blocking method as shown in Patent Document 2 is employed. In this coordinate input / detection device, two light emitting / receiving devices installed at both lower ends of the display unit 112 emit a plurality of infrared rays in parallel with the display unit 112 and are provided around the display unit 112. Light reflected by the reflecting member on the same optical path is received. The contact detection device 226 notifies the coordinate detection unit 224 of infrared identification information emitted by the two light emitting and receiving devices blocked by the object, and the coordinate detection unit 224 identifies the coordinate position that is the contact position of the object.

In another embodiment, a capacitive touch panel that identifies a contact position by detecting a change in capacitance, a resistive film type touch panel that identifies a contact position based on a voltage change between two opposing resistance films, and a contact Various detection means such as an electromagnetic induction type touch panel that detects an electromagnetic induction generated when an object contacts the display unit and identifies a contact position may be employed.
The coordinate detection unit 224 is a functional unit that calculates a coordinate position, which is a position where an object contacts the display unit 112, and issues various events. In the present embodiment, the coordinate detection unit 224 calculates the coordinate position of the contact position of the object by using the blocked infrared identification information notified by the contact detection device 226. The coordinate detection unit 224 issues various events to the event processing unit 210 together with the coordinate position of the contact position.
An event issued by the coordinate detection unit 224 is an event (TOUCH) for notifying that an object has been touched or approached, and that a contact point or an approach point has been moved while the object is in contact with or approaching the display unit 112. An event (MOVE) and an event (RELEASE) notifying that the object has left the display unit 112 are included. These events include coordinate position information that is contact position coordinates and approach position coordinates.
The drawing device 240 is a device that draws by making contact with the contact detection device 226 of the image processing device 110. The drawing device 240 has a pen shape provided with a contact detection device that detects contact of an object at the tip, and when the contact detection device comes into contact with the object, a contact signal indicating contact is identified by the drawing device. It transmits to the coordinate detection part 224 with information.

  In addition, the drawing apparatus 240 includes a mode switch for switching between the image processing apparatus operation mode or the user PC operation mode on the side surface, the rear end, and the like. The image processing apparatus operation mode is a mode in which the user can draw an arbitrary figure, character, or the like on the display unit 112 of the image processing apparatus 110, and can select an object such as a menu or a button displayed on the display unit 112. Mode. The user PC operation mode is a mode in which the user can select objects such as menus and buttons displayed on the display unit 112.

For example, when the drawing device 240 is brought into contact with the image processing device 110 while the user presses the mode switch, the drawing device 240 sets the user PC operation mode together with the contact signal and the identification information of the drawing device. The mode type signal shown is transmitted. When the drawing apparatus 240 is brought into contact with the image processing apparatus 110 without the user pressing the mode switch, the drawing apparatus 240 displays the image processing apparatus operation mode together with the contact signal and the identification information of the drawing apparatus. A mode type signal indicating is transmitted.
In the present embodiment, when the coordinate detection unit 224 receives infrared identification information from the contact detection device 226, the coordinate detection unit 224 calculates a coordinate position, which is a contact position of an object, and then receives a contact signal from the drawing device 240, thereby performing various events. Issue. At this time, the coordinate detection unit 224 notifies the event processing unit 210 of information indicating the mode type (hereinafter referred to as “mode type information”) together with the event.
In this embodiment, various signals are transmitted by near field communication such as Bluetooth (registered trademark). In other embodiments, various signals can be transmitted by wireless communication using ultrasonic waves or infrared rays.

The event processing unit 210 is a functional unit that processes an event issued by the coordinate detection unit 224. When the user PC operation mode is designated, when the event processing unit 210 receives an event from the coordinate detection unit 224, the event processing unit 210 transmits a position instruction event to the user PC 130a or the user PC 130b. When the image processing apparatus operation mode is designated, when receiving an event from the coordinate detection unit 224, the event processing unit 210 notifies the drawing instruction event and the selection notification event to other functional units of the image processing apparatus 110. .
The position instruction event is an event issued by an input device such as a mouse of the user PCs 130a and 130b, and is an event that indicates the position of the pointer. When the user PC operation mode is designated, it is issued to the user PCs 130a and 130b due to contact by the drawing device 240. The event processing unit 210 converts the coordinate position information included in the event issued by the coordinate detection unit 224 into coordinate position information corresponding to the screen size of the user PCs 130a and 130b, and transmits it to the user PCs 130a and 130b together with the position instruction event. The user PCs 130a and 130b process the position instruction event in the same manner as an event issued by an input device such as a mouse.

The drawing instruction event is an event that instructs the image processing apparatus 110 to perform drawing. The drawing instruction event is issued when the drawing apparatus 240 touches the display unit 112 when the image processing apparatus operation mode is designated.
The selection notification event is an event indicating that various objects such as buttons and menu bars constituting the screen displayed on the display unit 112 have been selected. The selection notification event is issued due to the drawing device 240 touching the display unit 112 when the image processing device operation mode is designated. The event processing unit 210 issues a selection notification event when the coordinate position information included in the event issued by the coordinate detection unit 224 is within the coordinate area of the object.
In this embodiment, identification information is assigned to each of the drawing instruction event and the selection notification event, and the functional unit of the image processing apparatus 110 that operates using these events as a trigger refers to the identification information. Execute the process. Further, identification information of the selected object is added to the selection notification event, and the functional unit of the image processing apparatus 110 that operates using the selection notification event as a trigger refers to the identification information of the object to perform various processing. Execute.

The drawing generation unit 216 is a functional unit that generates a drawing image drawn by the user using the drawing device 240. The drawing generation unit 216 generates an image layer in which the color of the coordinate position indicated by the coordinate position information is changed to a specific color. The drawing generation unit 216 stores the coordinate position as drawing information in the drawing information storage area (drawing information storage area 254) of the RAM 204. That is, the coordinate position (drawing position information) of the drawing image is stored as one piece of drawing information.
The synthesizing unit 218 is a functional unit that synthesizes various images. The composition unit 218 includes an image layer in which the application image generation unit 212 should draw an image (hereinafter referred to as “application image layer”), and an image layer in which the layout management unit 214 should draw the display images of the user PCs 130a and 130b. (Hereinafter referred to as “image capture layer”) and the image layer (hereinafter referred to as “handwriting layer”) to which the drawing generation unit 216 should draw an image are synthesized.

  The display control unit 220 is a functional unit that controls the display unit 112. The display control unit 220 displays the combined image generated by the combining unit 218 on the display unit 112. In the present embodiment, the composition unit 218 calls the display control unit 220 and displays the composite image on the display unit 112. In another embodiment, the combining unit 218 and the display control unit 220 may combine the image layers and display them on the display unit 112 at the same frequency as the update frequency of the image frame included in the image information.

The snapshot generation unit 222 is a functional unit that generates a snapshot image that is a composite image of the display images of the user PCs 130 a and 130 b and the drawing image generated by the drawing generation unit 216. When the snapshot generation unit 222 receives a selection notification event indicating that the snapshot button instructing acquisition of the snapshot displayed on the display unit 112 has been selected, the snapshot generation unit 222 combines the image capture layer and the handwriting layer, Generate a shot image. When generating the snapshot image, the snapshot generation unit 222 causes the repository management unit 228 to save the snapshot image in the storage device 230.
The repository management unit 228 is a functional unit that controls the storage device 230 that stores the snapshot image. As described above, the repository management unit 228 stores the snapshot image in the storage device 230 according to the instruction from the snapshot generation unit 222. Further, the repository management unit 228 acquires a snapshot image from the storage device 230 according to an instruction from the user PCs 130a and 130b, and transmits the snapshot image to the user PCs 130a and 130b.
Note that when the image processing apparatus 110 is activated, the snapshot image stored in the storage device 230 is deleted.

[First embodiment]
The image processing apparatus 110 operates under a plurality of operation modes (operation states), and the processor 200 controls the operation mode (operation state) of the image processing apparatus 110.
The operation mode includes a normal operation mode (non-power saving mode) in which all the components of the image processing system 100 are operated, and a standby mode (power saving) in which some functions of the image processing system 100 are stopped to reduce power consumption. Power mode). In the standby mode, power supply to at least the processor 200, the hard disk device 250, and the display unit 112 is stopped, and the function is stopped. Note that the power supply to the image acquisition unit 206 and the storage device 230 may be stopped in the standby mode to stop the function.
In the image processing system 100, when a user operation is not performed for a certain period of time, this is detected, and the normal operation mode is shifted to the standby mode.
In particular, the image processing apparatus 110 according to the present embodiment includes a hard disk device 250 as a nonvolatile storage unit that retains stored contents even when the power is turned off. The present embodiment is characterized in that the recovery data necessary for subsequently reproducing the work state at the time of transition to the standby mode is stored in the hard disk device 250 and then the transition to the standby mode is made.

The image processing system 100 has a timer 252 (software timer) as a program module that is developed and executed in the RAM 204. The timer 252 is a functional unit that counts the time after the user no longer performs an operation and issues a standby mode transition event to the processor 200 when a predetermined time is counted. When the coordinate detection unit 224 issues an event based on contact detection of an object by the contact detection device 226, that is, when an operation by the user is detected, the timer 252 resets the time being measured and starts a new time measurement. To do. The timer 252 issues a standby mode transition event to the processor 200 when measuring a predetermined time set in advance.
The time set in the timer 252 can be arbitrarily set by the user from, for example, an application setting menu displayed on the display unit 112 when the image processing system 100 is activated.

The timer 252 issues a standby mode transition event when a predetermined time has elapsed since the coordinate detection unit 224 acquired the coordinate position (position information) that is the contact position of the object. That is, the timer 252 measures a predetermined time after detecting the last contact of the object with the image processing apparatus, that is, the last operation with respect to the image processing apparatus.
The standby mode transition event is an event that instructs the processor 200 to transition to the standby mode. When the standby mode transition event is issued, the processor 200 reads out the snapshot image stored in the storage device 230 and the drawing position information stored in the drawing information storage area 254 of the RAM 204, and stores them in the hard disk device 250. After performing the storing process, control is performed to shift to the standby mode.
In addition, you may make it transfer to standby mode based on the instruction | indication from a user. That is, in the normal operation mode, when an object indicating standby is selected from a button, a menu bar, or the like constituting the screen displayed on the display unit 112, the event processing unit 210 sets the standby state to the processor 200. A mode transition event may be issued.
The return data is data including at least one of a snapshot image saved in the storage device 230 and drawing position information saved in the drawing information storage area 254 of the RAM 204. In the present embodiment, both the snapshot image and the coordinate position are stored as restoration data. The drawing position information indicates the coordinates of the drawing image drawn by the user using the drawing device 240, and is information for enabling the drawing image to be edited later.

FIG. 3 is a flowchart illustrating processing executed when the processor 200 of the image processing apparatus performs control for shifting from the normal operation mode to the standby mode.
In step S301, the processor 200 determines whether or not a standby mode transition event has been received from the timer 252. If processor 200 has not received the standby mode transition event (no in step S301), the processing in step S301 is continued. When the processor 200 receives the standby mode transition event (yes in step S301), the process of step S302 is performed.
In step S302, the processor 200 reads the snapshot image from the storage device 230, and causes the hard disk device 250 to store the snapshot image and the drawing position information stored in the drawing information storage area 254 of the RAM 204 as restoration data.

In step S303, the processor 200 performs a process of controlling to shift to the standby mode. By this processing, the functions of the processor 200, the hard disk device 250, and the display unit 112 are stopped.
Note that when the operation by the user is detected while the image processing apparatus 110 is in the standby mode, the image processing apparatus 110 returns from the standby mode. For example, when the contact detection device 226 detects contact of an object, the processor 200 wakes up using this as a trigger. The processor 200 performs a process of reading return data stored in the hard disk device 250. Further, the processor 200 stores the snapshot image in the return data in the storage device 230 and controls the drawing position information in the return data to be stored in the drawing information storage area 254 of the RAM 204.
As described above, by returning the information temporarily stored in the hard disk device 250 at the time of transition to the standby mode to the respective units stored before the transition to the standby mode, the image processing apparatus 110 changes the work state before the transition to the standby mode. Can be reproduced.

As described above, according to the present embodiment, the recovery data is stored in the hard disk device 250 before shifting to the standby mode. Therefore, when shifting to the normal operation mode, the working state when shifting to the standby mode is restored. be able to. Further, when the image processing apparatus 110 is turned off after the transition to the standby mode, or when the standby mode is restored to the normal operation mode, the snapshot image in the storage device 230 and the information in the drawing information storage area 254 are stored. Even if it is erased artificially, it is possible to reproduce the work state before the transition to the standby mode by restoring each information using the recovery data read from the hard disk device 250.
When the timer 252 counts that a predetermined time has elapsed since the coordinate detection unit 224 acquired the coordinate position (position information), an event for causing the hard disk device to store return data from the timer 252 to the processor. 200 may be issued.

[Second Embodiment]
An image processing system according to the second embodiment of the present invention will be described.
In the first embodiment, both the snapshot image and the drawing position information are unconditionally stored in the hard disk device 250 when shifting to the standby mode. The present embodiment is characterized in that either or both of the snapshot image and the drawing position information are selectively stored in the hard disk device in accordance with the state of setting information for setting data to be included in the return data. .
The image processing apparatus 110 includes setting information (not shown) in which data to be included in the return data is set. The setting information is stored in the hard disk device 250, for example. In the setting information, information on whether or not to include the snapshot image and / or drawing position information in the restoration data is recorded.
The content of the setting information is arbitrarily specified by the user from an application setting menu displayed on the display unit 112 when the image processing system 100 is activated, for example. The setting information is read from the hard disk device 250 when the image processing system 100 is started up and stored in a setting information storage area in the RAM 204.

When storing the recovery data in the hard disk device 250, the processor 200 generates the recovery data according to the contents recorded in the setting information, stores the data in the hard disk device 250, and then shifts to the standby mode.
As described above, according to the present embodiment, only the information necessary as restoration data is stored in the hard disk device 250 according to the state of the setting information, so that the storage area required for the hard disk device 250 can be reduced.

[Third embodiment]
An image processing system according to the third embodiment of the present invention will be described. The image processing system according to the present embodiment is characterized in that data included in return data is automatically determined.
As shown in FIG. 2, the image processing apparatus 110 includes a data output interface 258 as a physical interface for outputting data to a flash memory 256 which is an example of an external storage unit. In the present embodiment, a USB socket can be adopted as the data output interface 258.
When an object indicating saving to a flash memory is selected from objects such as menus and buttons displayed on the display unit 112 and a snapshot image is selected as a data type to be saved, the repository management unit 228 stores A snapshot image is acquired from the device 230 and transmitted to the flash memory 256.
The image processing apparatus 110 includes an output information recording area 260 for recording information related to data output. The output information recording area 260 is an area provided in the RAM 204. The output information includes the latest time when data is output to the flash memory 256 and the type information of the output data. The repository management unit 228 records output information in the output information recording area 260 when outputting data to the flash memory 256.

Further, the image processing apparatus 110 includes a generation time recording area 262 for recording the latest time when the drawing generation unit 216 generates a drawing image. The generation time recording area 262 is an area provided in the RAM 204 for recording the generation time.
In addition, when the snapshot image is stored, the storage device 230 stores the snapshot image and its storage time in association with each other.

A method of determining information to be included in the return data will be described based on the flow of FIG. FIG. 4 is a flowchart showing a process for determining information to be included in the return data. Hereinafter, an example of determining whether to include drawing position information in the return data will be described. Note that the same steps as those in the flowchart shown in FIG.
When receiving the standby mode transition event (yes in step S301), the processor 200 performs the process of step S401.
In step S401, the processor 200 compares the storage time of the snapshot image in the storage device 230 with the generation time of the drawing image. That is, the processor 200 compares the snapshot image storage time in the storage device 230 with the drawing image generation time recorded in the generation time recording area 262. If the storage time of the snapshot image is new (no in step S401), the process ends. If the generation time of the drawing image is new (yes in step S401), the hard disk device 250 is controlled to store the drawing position information as return data (step S402).
As described above, in the present embodiment, when the drawing image is generated after the snapshot image is stored in the storage device 230, the processor 200 controls to include the drawing position information in the return data. By performing this control, the latest drawing position information can be stored in the hard disk device 250.

When determining whether to include the snapshot image in the restoration data, the processor 200 outputs the snapshot image output time recorded in the output information recording area 260 and the snapshot image stored in the storage device 230. Is compared with the stored time. When the storage time is new, the processor 200 reads the snapshot image stored in the storage device 230, generates restoration data including the snapshot image, and controls the hard disk device 250 to store it.
As described above, in this embodiment, when a new snapshot image is stored in the storage device 230 after the snapshot image is output to the flash memory 256, the processor 200 includes the snapshot image in the restoration data. To control. By performing this control, the latest snapshot image can be stored in the hard disk device 250.

As described above, according to the present embodiment, the processor determines the data contents to be included in the return data, so that the operation burden on the user can be reduced. In addition, when it can be confirmed that the latest data is stored, it is possible to prevent the data from being included in the restoration data, so that the area required for data storage on the hard disk can be reduced.
The drawing position information may be output to the flash memory 256. In this case, the output information recording area 260 includes the time when the data is output to the flash memory 256 and the fact that the drawing position information is output as the type information of the output data. Then, the processor 200 compares the storage time of the drawing position information in the flash memory with the generation time of the drawing image, and controls to include the drawing position information in the return data when the drawing image generation time is new. .

[Fourth embodiment]
An image processing system according to a fourth embodiment of the present invention will be described with reference to FIG. The image processing system according to the present embodiment is characterized in that user information is stored in association with return data.
The image processing apparatus 110 includes a user data acquisition unit 264 and a user information management unit 266.
The user data acquisition unit 264 (user information acquisition means) is means for acquiring data related to the user who uses the image processing apparatus 110. The user data acquisition unit 264 can be, for example, a sensor such as an IC card reader, a fingerprint recognition unit, a face recognition unit, or a voice recognition unit. The user data acquisition unit 264 transmits the acquired user data to the user information management unit 266. The user data referred to here is the contents registered in the IC card, or physical features of the user such as fingerprints, faces, and voices.
When the user data acquisition unit 264 is a face recognition unit or a voice recognition unit, the user data acquisition unit 264 can acquire user data during an electronic conference using the image processing system. For example, when the user data acquisition unit 264 is a face recognition unit, it captures a pattern of an electronic conference, extracts all face areas detected from the image, and calculates a facial feature amount for each face area. Data of a plurality of users can be acquired.

The user information management unit 266 is one of program modules that are loaded into the RAM 204 and executed, and is a functional unit that manages user information. The user information management unit 266 identifies a user (user name) based on the user data transmitted by the user data acquisition unit 264, and manages the identified user information. The user information management unit 266 stores user information in the user information storage area 268 of the RAM 204. The user information includes at least a user name.
When the user data acquisition unit 264 is a fingerprint recognition unit, a face recognition unit, a voice recognition unit, or the like, the user (user name) cannot be specified only by the user data acquired by the user data acquisition unit 264. In this case, a user database in which the physical features of the user and the user name are associated and stored in advance is prepared separately. The user information management unit 266 compares the physical feature amount transmitted from the user data acquisition unit 264 with the physical feature amount registered in the user database. When data having a physical feature amount similarity greater than or equal to a predetermined value is found from the user database, a user registered in association with the physical feature amount is identified as a user of the image processing system.

  The user information management unit 266 stores at least a user name in the user information storage area 268 as user information. The user information stored in the user information storage area 268 may include the physical feature amount of the user acquired by the user data acquisition unit. Further, it may include a password (authentication information) used when the contents of the return data are expanded in each part of the image processing apparatus. This password may be arbitrarily registered when registering the user information, or may be registered in advance in the user database. Instead of the password, other authentication information for confirming the identity of the user may be used.

A flow from user data acquisition by the user data acquisition unit to user identification by the user information management unit and storage of user information will be described.
For example, after the image processing system 100 is activated, the application image generation unit 212 draws an initial menu screen on the application image layer. The user information registration process is started by selecting a user information registration menu from the initial menu displayed on the display unit 112. The application image generation unit 212 causes the display unit 112 to display a message that prompts the user data acquisition unit 264 to read the user data.
When receiving the user data acquired by the user data acquisition unit 264, the user information management unit 266 specifies the user based on the user data, and stores the specified user information in the user information storage area 268.

The transition from the normal operation mode to the standby mode in this embodiment will be described based on the flowchart of FIG. FIG. 5 is a flowchart showing processing when the image processing apparatus according to the fourth embodiment of the present invention shifts from the normal operation mode to the standby mode. Note that the same steps as those in the flowchart shown in FIG.
When receiving the standby mode transition event (yes in step S301), the processor 200 performs the process of step S501.
In step S501, the processor 200 confirms whether the user information is stored in the user information storage area 268. If the user information is stored (yes in step S501), the process of step S502 is performed. If the user information is not stored (no in step S501), the process of step S503 is performed.
In step S502, the processor 200 reads the user information from the user information storage area 268, and stores the read user information and return data in the hard disk device 250 in association with each other.
In step S503, the processor 200 stores only the return data in the hard disk device 250.

The transition from the standby mode to the normal operation mode will be described.
When returning from the standby mode to the normal operation mode, first, the user information storage area 268 is initialized, and user information re-registration processing is performed. In other words, the user information management unit 266 erases user information stored in the user information storage area 268 and initializes the user information storage area 268. The user information management unit 266 causes the display unit 112 to display a message that prompts the user data acquisition unit 264 to read user data. When receiving the user data acquired by the user data acquisition unit 264, the user information management unit 266 specifies the user based on the user data, and stores the specified user information in the user information storage area 268.

After the above processing is performed, restoration data reading processing by the processor 200 is performed. FIG. 6 is a flowchart showing a process when shifting from the standby mode to the normal operation mode.
In step S <b> 601, the processor 200 confirms whether or not return data associated with user information matching the user information stored in the user information storage area 268 exists in the hard disk device 250. If there is no corresponding return data (no in step S601), the process ends without reading the return data. If the corresponding return data exists (YES in step S601), the processor 200 performs the process of step S602.
In step S602, the processor 200 controls the display unit 112 to display a screen that allows the user to select whether or not there is return data that matches the user information and whether or not to read the return data.
In step S603, the processor 200 confirms whether or not reading of return data is selected. If it is selected not to read the return data (NO in step S603), the process ends without reading the return data. If reading of the return data is selected (yes in step S603), the process of step S604 is performed.

At step S604, the processor 200 confirms whether or not a password (authentication information) exists in the user information. If the password does not exist (No in step S604), the process of step S607 is performed. If there is a password (YES in step S604), the processor 200 performs the process of step S605.
In step S605, the processor 200 controls the display unit 112 to display a screen that prompts the user to input a password.
At step S606, the processor 200 confirms whether or not the input password matches the password in the user information. If the passwords do not match (NO in step S606), the process ends without reading the return data. If the passwords match (Yes in step S606), the processor 200 performs the process of step S607.
In step S <b> 607, the processor 200 reads the restoration data from the hard disk device 250, saves the snapshot image in the restoration data in the storage device 230, and saves the drawing position information in the restoration data in the drawing information storage area 254. Control to do.

As described above, according to the present embodiment, the image processing apparatus can determine whether or not the users before and after the transition to the standby mode match by storing the user information in association with the return data. The image processing apparatus performs a process of presenting the return data stored before the transition to the standby mode to the user only when the users match, and does not present the return data to the user when the users do not match. It can be performed. If the users of the image processing apparatus are different before and after the transition to the standby mode, the subsequent users will not receive the presentation of unnecessary data, and the user's operation burden and time loss will be reduced without being forced to perform unnecessary operations. I can plan.
Further, by adding authentication information such as a password to the return data, the contents of the return data cannot be confirmed if the authentication information does not match. Accordingly, there is no possibility that the contents of the return data can be seen by users other than those concerned, and the security of the conference information using the image processing apparatus and the image processing system is maintained.

[Other aspects of image processing system]
FIG. 7 is a diagram showing another embodiment of the image processing system of the present invention. Hereinafter, with reference to FIG. 7, the image processing system 1000 will be described focusing on differences from the image processing system 100 shown in FIG.
In the image processing system 1000, an image processing apparatus 1010a, an image processing apparatus 1010b, and an image processing apparatus 1010c are connected via a network 1014. The network 1014 is a network such as a LAN or the Internet, and communicates various data between the image processing apparatuses 1010a, 1010b, and 1010c.
The image processing apparatus 1010 a is connected to user PCs 130 a and 130 b via cables 124 and 126. The image processing apparatus 1010b is connected to a user PC 130c via cables 124 and 126. The image processing apparatuses 1010a, 1010b, and 1010c communicate information such as image data and events with each other via the network 1014.

  In the embodiment shown in FIG. 7, the image processing apparatuses 1010a, 1010b, and 1010c are connected via the network 1014. However, in other embodiments, these image processing apparatuses are not connected via the network 1014, such as a star type. You may connect directly with a connection form.

FIG. 8 is a diagram illustrating a hardware configuration and a functional configuration of the image processing apparatus according to the embodiment illustrated in FIG. 7. Hereinafter, the hardware configuration and functional configuration of the image processing apparatus 1010a will be described with a focus on differences from the image processing apparatus 110 with reference to FIG. Note that the hardware configuration and functional configuration of the image processing apparatuses 1010a to 1010b are the same as those of the image processing apparatus 1010a, and a description thereof will be omitted.
The image processing apparatus 1010a includes a communication control unit 270 and a communication unit 272. The communication control unit 270 is a functional unit that controls communication between image processing apparatuses via the network 1014. The communication unit 272 is a network interface with the network 1014. The communication control unit 270 communicates information such as authentication information, image data such as image frames and snapshot images, drawing information, and events via the communication unit 272.
When the standby mode transition event occurs in the image processing apparatus 1010a constituting the image processing system 1000, the image processing apparatus 1010a shifts from the normal operation mode to the standby mode. In the case of this embodiment, as a trigger for the image processing apparatus 1010a to return to the normal operation mode, it is possible to use data communication from another image processing apparatus 1010b in addition to the object contact detection by the contact detection apparatus 226. it can.

[Example Embodiment]
The present invention can be implemented in the following modes.
(Aspect 1)
An image processing apparatus 110 that displays an image on a display unit (display unit 112), and that generates a drawing image using position information indicating a contact position or an approach position of an object to the display unit (drawing generation unit) 216), image acquisition means (image acquisition unit 206) for acquiring a display image from the information processing apparatus (user PC 130), and snapshot storage means (storage device) for storing a snapshot image obtained by combining the display image and the drawing image 230), nonvolatile storage means (hard disk device 250) that retains the stored contents even when the power is shut off, and control to switch the operation mode to a power saving mode (standby mode) or a non-power saving mode (normal operation mode) Control means (processor 200) for controlling the operation mode from the non-power saving mode to the power saving mode. Before the control to switch to the mode, the non-volatile storage means stores return data including at least one of the drawing position information indicating the coordinate position of the drawing image and the snapshot image read from the snapshot storage means. Control.
In this aspect, before switching the operation mode from the non-power saving mode to the power saving mode, at least one of the drawing position information and the snapshot image is stored in the nonvolatile storage means. In the image processing apparatus, the drawing position information and the snapshot image are one piece of information that enables reproduction of the content displayed on the display means. Therefore, even when the power supply of the image processing apparatus is turned off by mistake, the content displayed on the display means can be reproduced.

(Aspect 2)
An image processing apparatus 110 that displays an image on a display unit (display unit 112), and that generates a drawing image using position information indicating a contact position or an approach position of an object to the display unit (drawing generation unit) 216), image acquisition means (image acquisition unit 206) for acquiring a display image from the information processing apparatus (user PC 130), and snapshot storage means (storage device) for storing a snapshot image obtained by combining the display image and the drawing image 230), nonvolatile storage means (hard disk device 250) that retains the stored contents even when the power is shut off, and control to switch the operation mode to a power saving mode (standby mode) or a non-power saving mode (normal operation mode) Control means (processor 200) for measuring time, and a timer 252 for measuring time. Data for restoration including at least one of drawing position information indicating the coordinate position of the drawing image and the snapshot image read from the snapshot storage means when the timer counts that a predetermined time has elapsed since the last operation on the apparatus Is controlled to be stored in the nonvolatile storage means.
In this aspect, when the timer counts that a predetermined time has elapsed since the last operation on the image processing apparatus, at least one of the drawing position information and the snapshot image is stored in the nonvolatile storage means. In the image processing apparatus, the drawing position information and the snapshot image are one piece of information that enables reproduction of the content displayed on the display means. Therefore, even when the image processing apparatus is no longer operated and switched to the power saving mode, the content displayed on the display unit can be reproduced even when the power of the image processing apparatus is accidentally turned off. be able to.

(Aspect 3)
The control means (processor 200) stores the drawing position information and / or the restoration data including the snapshot image in accordance with the state of the setting information for setting the data to be included in the restoration data. ) Is controlled so that it is memorized.
In this aspect, the setting information is data to be included in the return data, and information on whether or not to include both the snapshot image and the drawing position information in the return data is recorded. According to this aspect, since only the information necessary as restoration data is stored in the hard disk device 250, the storage area required for the hard disk device 250 can be reduced.

(Aspect 4)
A generation time recording unit (generation time recording area 262) for recording the generation time of the drawing image is provided, and the control unit (processor 200) includes the generation time of the drawing image recorded in the generation time recording unit and the snapshot storage unit ( The time when the snapshot image is stored in the storage device 230) is compared, and when the generation time of the drawing image is new, the return data including the drawing position information is stored in the nonvolatile storage means (hard disk device 250). To control.
In this aspect, when the drawing image is generated after the snapshot image is stored in the storage device 230, the processor 200 controls to include the drawing position information in the return data. By performing this control, the latest drawing position information can be stored in the hard disk device 250. In addition, since the processor determines the data contents to be included in the return data, the operation burden on the user can be reduced. When it is confirmed that the latest data is stored, the data can be excluded from the restoration data, so that the area required for data storage on the hard disk can be reduced.

(Aspect 5)
Output information recording means for recording the time when the snapshot image is output to the external storage means is provided, and the control means (processor 200) outputs the time when the snapshot image was stored in the snapshot storage means (storage device 230) and the output. The output time of the snapshot image recorded in the information recording means is compared. When the storage time of the snapshot image is new, the restoration data including the snapshot image is stored in the non-volatile storage means (hard disk device 250). To control.
In this aspect, when the drawing image is generated after the snapshot image is stored in the storage device 230, the processor 200 controls to include the drawing position information in the return data. By performing this control, the latest drawing position information can be stored in the hard disk device 250. In addition, since the processor determines the data contents to be included in the return data, the operation burden on the user can be reduced. When it is confirmed that the latest data is stored, the data can be excluded from the restoration data, so that the area required for data storage on the hard disk can be reduced.

(Aspect 6)
User information acquisition means (user data acquisition unit 264) for acquiring user information is provided, and the control means (processor 200) controls the user information to be associated with the return data and stored in the nonvolatile storage means.
In this aspect, it is possible to confirm whether or not the users before and after the transition to the standby mode match by storing the user information in association with the return data.

(Aspect 7)
The control means (processor 200) includes drawing information storage means (drawing information storage area 254) for storing drawing position information and user information storage means (user information storage area 268) for storing user authentication information in advance. When the operation mode is controlled to be switched from the power saving mode to the non-power saving mode, the authentication information stored in the user information storage unit regarding the user associated with the return data, and the authentication information input from the user If the two match, the return data is read from the non-volatile storage means (hard disk device 250), the drawing position information included in the return data is stored in the drawing information storage means, and the snapshot image included in the return data is stored. Is controlled to be stored in the snapshot storage means (storage device 230).
In this aspect, if the authentication information does not match, the contents of the return data cannot be confirmed. Accordingly, there is no possibility that the contents of the return data can be seen by users other than those concerned, and the security of the conference information using the image processing apparatus and the image processing system is maintained.

(Aspect 8)
An image processing method executed by an image processing apparatus that displays an image on a display unit, wherein the drawing generation unit generates a drawing image using position information indicating a contact position or an approach position of an object with respect to the display unit; The image acquisition means acquires the display image from the information processing apparatus, the snapshot storage means stores the snapshot image obtained by combining the display image and the drawing image, and the control means saves the operation mode. Control to switch to the power mode or the non-power saving mode, and the control means includes at least one of drawing position information indicating the coordinate position of the drawing image and a snapshot image read from the snapshot storage means. Data is stored in non-volatile storage means that retains the stored contents even when the power is turned off. Controlling the mode from the non-power saving mode to switch to the power saving mode.
This aspect is an image processing method corresponding to aspect 1, and has the same operations and effects as aspect 1.

(Aspect 9)
A program for causing an image processing apparatus to execute the image processing method according to aspect 8.
This aspect is a program for causing an image processing apparatus to execute an image processing method corresponding to aspect 8, and has the same operations and effects as aspect 8.

(Aspect 10)
An image processing system for displaying an image, using image acquisition means for acquiring a display image from an information processing device, display means for displaying an image, and position information indicating a contact position or an approach position of an object with respect to the display means A drawing generation unit that generates a drawing image, a snapshot storage unit that stores a snapshot image obtained by combining the display image and the drawing image, a non-volatile storage unit that retains the stored contents even when the power is shut off, and an operation mode. Control means for controlling the operation mode to be switched from the non-power-saving mode to the power-saving mode. Return data including at least one of drawing position information indicating the position and a snapshot image read from the snapshot storage means The controls to be stored in non-volatile memory means.
This aspect is an image processing system corresponding to aspect 1, and has the same operations and effects as aspect 1.

  DESCRIPTION OF SYMBOLS 100 ... Image processing system, 110 ... Image processing apparatus, 112 ... Display part, 124, 126 ... Cable, 130 ... User PC, 200 ... Processor (control means), 202 ... ROM, 204 ... RAM, 206 ... Image acquisition part, 208 ... Video RAM, 210 ... Event processing unit, 212 ... Application image generation unit, 214 ... Layout management unit, 216 ... Drawing generation unit, 218 ... Composition unit, 220 ... Display control unit, 222 ... Snapshot generation unit, 224 ... Coordinate detection unit, 226 ... contact detection device, 228 ... repository management unit, 230 ... storage device (snapshot storage means), 232 ... image input interface, 234 ... image output interface, 240 ... drawing device, 250 ... hard disk device (non-volatile) Storage means), 252 ... timer, 254 ... drawing information storage area, 56 ... Flash memory, 258 ... Data output interface, 260 ... Output information recording area, 262 ... Generation time recording area, 264 ... User data acquisition section (user information acquisition means), 266 ... User information management section, 268 ... User information storage Area, 270 ... Communication control section, 272 ... Communication section, 1000 ... Image processing system, 1010 ... Image processing apparatus, 1014 ... Network

JP 2013-41458 A

Claims (6)

An image processing apparatus for displaying an image on a display means,
Drawing generation means for generating a drawing image using position information indicating a contact position or an approach position of an object to the display means ;
Snapshot storage means for storing a snapshot image obtained by synthesizing the display image supplied from the outside and the drawing image;
Non-volatile storage means for retaining stored contents even when the power is shut off;
Control means for controlling the operation mode to switch to the power saving mode or the non-power saving mode;
With
The control means, before controlling the operation mode to switch from the non-power saving mode to the power saving mode, sets the coordinate position of the drawing image according to the state of the setting information for setting the data to be included in the return data. the image processing apparatus characterized by controlling at least one of the snapshot image drawing position information read from the snapshot storage means shown so as to be stored in the nonvolatile storage means included in the return data. Before SL control means, the image processing apparatus according to claim 1, wherein the controller controls so that the to be stored in the nonvolatile storage means in association with User chromatography The information in the return data. Drawing information storage means for storing the drawing position information;
User information storage means for storing user authentication information in advance,
The control means, when controlling the operation mode to switch from the power saving mode to the non-power saving mode, the authentication information stored in the user information storage means regarding the user associated with the return data, and the user the drawing information the drawing position information when the authentication information input when there is a match, reads the recovery data from said nonvolatile memory means includes said drawing position information on the return data from the 3. The image according to claim 2 , wherein the image is stored in a storage unit, and when the snapshot image is included in the restoration data, the snapshot image is controlled to be stored in the snapshot storage unit. Processing equipment. An image processing method executed by an image processing apparatus that displays an image on a display means,
A drawing generation unit generating a drawing image using position information indicating a contact position or an approach position of an object to the display unit ;
A step of snapshot storage means stores a snapshot image obtained by synthesizing the said drawn image with the supplied display image from the outside,
Controlling the control means to switch the operation mode to a power saving mode or a non-power saving mode,
The control unit is configured to output at least one of drawing position information indicating a coordinate position of the drawing image and a snapshot image read from the snapshot storage unit according to a state of setting information for setting data to be included in return data. The image is characterized in that it is included in the return data and stored in a non-volatile storage means that retains the stored contents even when the power is shut off, and the operation mode is controlled to be switched from the non-power saving mode to the power saving mode. Processing method. A program for causing an image processing apparatus to execute the image processing method according to claim 4 . An image processing system for displaying an image ,
Display means for displaying images,
Drawing generation means for generating a drawing image using position information indicating a contact position or an approach position of an object to the display means;
Snapshot storage means for storing a snapshot image obtained by synthesizing the display image supplied from the outside and the drawing image;
Non-volatile storage means for retaining stored contents even when the power is shut off;
Control means for controlling the operation mode to switch to the power saving mode or the non-power saving mode;
With
The control means, before controlling the operation mode to switch from the non-power saving mode to the power saving mode, sets the coordinate position of the drawing image according to the state of the setting information for setting the data to be included in the return data. image processing system and controlling at least one of the snapshot image read from the snapshot storage means and drawing position information indicating as to be stored in the nonvolatile storage means included in the return data.

Images