JP2013126153A - Capture image recording device and capture image recording method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable efficient recording of a capture image of a display screen.SOLUTION: Timing for recording a capture screen is controlled in accordance with the content of a screen change or a user operation for a display screen, thereby enabling prevention of inconvenience such as an increase in the amount of recorded data due to recording of many unimportant images, or inability to read characters due to reduction in resolution.

Description

  The present invention relates to a captured image recording apparatus and a captured image recording method for recording a captured image obtained by capturing a display screen.

In a conference system or the like, a displayed screen is often left as a conference record. At this time, it is very troublesome for the operator to manually capture the screen and save the image. There is also a problem that the displayed screen may be forgotten.
On the other hand, if all the display screens are recorded as moving images, there is a problem that the recording data volume becomes enormous because recording is always performed.

In view of this, a method has been proposed for recording a display screen automatically and as efficiently as possible. For example, the recording apparatus described in Patent Document 1 discloses a technique for recording the difference image only when the display screen changes.
In addition, the terminal monitoring device described in Patent Document 2 discloses a technique for recording a screen when an operation is performed by an operator and reducing the recording capacity according to the degree of risk evaluated for the operation. Yes.

JP 2011-28563 A JP 2008-193491 A

  However, in Patent Literature 1 and Patent Literature 2, when screen recording is performed for the purpose of looking back on the screen displayed during the meeting, efficient recording cannot be performed. For example, when an operator inputs characters using a keyboard, operations and screen changes occur continuously, which causes a problem that many unimportant images such as a screen during character input are recorded.

In addition, the terminal monitoring device described in Patent Document 2 performs capacity reduction such as reducing the resolution for an image determined to have a low risk level, but this method is suitable for monitoring unauthorized operations. . However, when it is used for reviewing meetings, the resolution of necessary materials is low, and there is a possibility that characters cannot be read.
In view of the above-described problems, an object of the present invention is to enable efficient recording of a captured image on a display screen.

  The captured image recording apparatus of the present invention is a captured image recording apparatus for recording a display screen, the operation event detecting means for detecting a user operation event, the screen change detecting means for detecting a change in the display screen, Based on the operation event detected by the operation event detecting means or the screen change detected by the screen change detecting means, the threshold of the display screen duration until the capture image recording means records the captured image on the display screen Threshold value determining means for determining the display screen, a display screen duration measuring means for measuring the time that the display screen continues without changing, and a display screen duration measured by the display screen duration measuring means The capture is performed so that a captured image of the display screen is recorded when the threshold determined by the determination unit is exceeded. And having a control means for controlling the image recording means.

  According to the present invention, a captured image of a display screen can be efficiently recorded.

It is a figure which shows the system configuration | structure of the captured image recording device of embodiment. It is a program block diagram with which the captured image recording apparatus of embodiment is provided. It is a figure which shows the characteristic of the relationship between the importance used for determination of a threshold value, and a threshold value. It is a flowchart explaining the flow of a process of a captured image recording device. It is a flowchart which shows the flow of the importance calculation process which an importance calculation means performs. It is a flowchart which shows the flow of the importance calculation process of a text input. It is a flowchart which shows the flow of the importance calculation process of a moving image / animation. It is a flowchart which shows the flow of the importance calculation process of another classification. It is a figure which shows an example of the value of the importance with respect to the specific operation and change for every classification | category.

Hereinafter, the present invention will be described in detail based on preferred embodiments with reference to the accompanying drawings. The configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.
<First Embodiment>
FIG. 1 is a block diagram illustrating a system configuration of a captured image recording apparatus 100 according to the first embodiment.
In FIG. 1, reference numeral 101 denotes a CPU that controls each apparatus, and executes a program stored in a memory. Reference numeral 102 denotes a memory for storing processing data and programs, which are read and written by the CPU 101.

  Reference numeral 103 denotes a storage device that assists the memory 102 and stores various applications and data files. An input control unit 104 controls input devices such as the keyboard 10 and the pointing device 20, receives operation information from the input device, and executes a process according to the operation information. Reference numeral 105 denotes an output control unit that performs output control when the CPU 101 outputs a calculation result, image data, and the like to the display panel 30.

  Note that the memory 102 and the storage device 103 may include an HDD, a flash memory, or a removable storage device in addition to the RAM and the ROM. In FIG. 1, the keyboard 10 and the pointing device 20 are described as examples of input devices for the input control unit 104, but a remote controller or the like may be used as the input device. A configuration in which a microphone or a camera is connected to the input control unit 104 is also possible. The output control unit 105 can output not only to the display panel but also to an external display device such as a television device or a projector device. Further, an output device such as a speaker may be connected to the output control unit 105.

  FIG. 2 is a diagram showing a program configuration provided in the captured image recording apparatus 100 of the present embodiment. The captured image recording apparatus 100 of this embodiment includes an operation event detection unit 201, a screen change detection unit 202, an importance level calculation unit 203, a threshold value determination unit 204, a display screen duration measurement unit 205, and an image recording unit 206. .

  The operation event detection unit 201 detects an operation event that occurs when a user operation is performed using an input device such as the keyboard 10 or the mouse 20. When an operation event is detected, the operation event detection unit 201 acquires information on the operation.

The screen change detection unit 202 detects that the content of the display screen output from the output control unit 105 has changed. When a screen change is detected, the screen change detection unit 202 acquires information on the screen change.
The importance calculation unit 203 calculates the importance based on the operation information detected and acquired by the operation event detection unit 201 or the screen change information detected and acquired by the screen change detection unit 202. In the present embodiment, the importance is 10 levels of 1 to 10, and 1 is the lowest value and 10 is the highest value, but the definition method of the number of importance levels and the magnitude relationship is limited to this method. Is not to be done.

Based on the importance calculated by the importance calculation unit 203, the threshold determination unit 204 determines a threshold for the display screen duration until the capture image is recorded.
The display screen duration measurement unit 205 measures the time during which the screen display state continues after an operation or screen change is detected, that is, the elapsed time since the previous operation or screen change was detected.

  The image recording unit 206 records a captured image when the time measured by the display screen duration measurement unit 205 reaches the threshold value determined by the threshold value determination unit 204. However, in the present embodiment, when the capture image to be recorded is the same as the previous recorded image, the recording is not performed.

  FIG. 3 is a diagram showing the characteristic of the relationship between the importance and the threshold used for the threshold determination in the threshold determination unit 204. The threshold is increased as the importance is lower, and the threshold is decreased as the importance is higher. By having the relationship between the importance level and the threshold value, the captured image is relatively difficult to be recorded when the importance level is low, and conversely, the captured image is relatively easily recorded when the importance level is high. In the present embodiment, the relationship between the importance level and the threshold value may be linear or non-linear. Further, the threshold value may be determined by dynamically changing the relationship between the importance level and the threshold value.

FIG. 4 is a flowchart showing a flow of processing executed by developing a program included in the captured image recording apparatus 100 according to the present embodiment in the memory 102.
First, in S401, it is determined whether or not to continue the process. If the process is to be continued, the process proceeds to S402. If it is instructed to end the process by a user operation or the like, the process is terminated.

  In S402, the operation event detection unit 201 and the screen change detection unit 202 monitor the presence or absence of a user operation and a screen change. In step S403, it is determined whether an operation event or a screen change is detected as a result of monitoring. If it is determined that an operation event or a screen change has been detected, the process proceeds to S404. If it is determined that an operation event or a screen change has not been detected, the process proceeds to S407.

  In step S404, the importance level calculation unit 203 calculates the importance level based on the operation information or the screen change information. In step S405, the threshold value determination unit 204 determines a threshold value for the display screen duration time until the capture image is recorded based on the importance level calculated in step S404. In step S406, the display screen duration measurement unit 205 starts measuring the display screen duration. If measurement has already been performed, the time is reset and measurement is newly started.

  In S407, it is determined whether or not the display screen duration time is measured by the display screen duration measurement unit 205 and the measured display screen duration has reached the threshold value determined in S405. If yes, the process proceeds to S408. If the measured display screen duration has not reached the threshold, or if the display screen duration has not been measured, the process proceeds to S401. In step S <b> 408, the image recording unit 206 records and saves the captured image in the storage device 103. In step S409, the display screen duration measurement unit 205 stops measuring the display screen duration. Thereafter, the process returns to S401 and the above-described processing is performed.

  In this embodiment, the display screen duration measurement unit 205 measures the display screen duration based on the time when the operation event or the screen change is detected. However, any method that can measure the display screen duration can be used. It is not limited to this method. For example, it is possible to record a time stamp at the time when an operation event or a screen change is detected, and to measure the elapsed time from the difference between the time stamp and the current time.

FIG. 5 is a flowchart showing the flow of importance calculation processing performed by the importance calculation unit 203 in S404 of FIG.
First, in step S <b> 501, the importance level calculation unit 203 acquires operation information detected by the operation event detection unit 201 or screen change information detected by the screen change detection unit 202. Subsequently, in S502, the contents of the active window are analyzed and classified. In the present embodiment, the contents of the window are classified into text input, moving image / animation, and others based on user operations and screen changes before and after. When the screen changes slightly according to the keyboard input operation, the window is classified as text input.

  Further, when the screen continues to change even if there is no user operation (event), the window is classified as a moving image / animation. If neither is true, classify as Other. Note that the classification and classification method are not limited to these. For example, classification such as graphic drawing or spreadsheet may be defined, or classification may be performed for each specific application.

  In step S503, it is determined whether the active window classification is text input. If it is determined that the input is a text input, the process proceeds to S504, and in S504, an importance calculation process in the case of text input described later is performed. If it is not determined in step S503 that the input is text, the process proceeds to step S505.

  In S505, it is determined whether or not the active window is classified as a moving image / animation. If it is determined that it is a moving image / animation, the process advances to step S506, and importance calculation processing in the case of a moving image / animation described later is performed in step S506. If it is not determined in S505 that the movie / animation is selected, the process proceeds to S507. In S507, importance calculation processing in the case of other classification described later is performed.

FIG. 6 is a flowchart showing the flow of importance calculation processing in the case of text input performed in S504 of FIG.
In step S601, it is determined whether the active window application has lost the focus. If it is determined that the focus has been lost, the process proceeds to S602. Otherwise, the process proceeds to S603.

  In S602, a value determined in advance for the operation (event) that the application loses focus is adopted as the result of the importance calculation. The importance value for a specific operation or change for each classification, such as focus out in the case of text input, is determined individually. An example of the importance in the case of performing individual threshold determination processing is shown in FIG. In the case of the example of FIG. 9, the importance is set to 10 in S602.

  In S603, it is determined whether or not the Enter key has been pressed. If it is determined that the Enter key has been pressed, the process proceeds to S604. Otherwise, the process proceeds to S605. In S604, a predetermined value for the operation (event) of pressing the Enter key is adopted as a result of the importance calculation. In the example of FIG. 9, the importance is set to 8 in S604.

  In S605, importance calculation processing in the case of other classifications described later is performed. Although the classification is text input, since an important operation unique to the text input has not been performed, the importance calculation processing similar to that for other classifications is performed. This process is the same as the process performed in S507 of FIG.

FIG. 7 is a diagram illustrating an example of importance calculation processing in the case of the moving image / animation performed in S506 of FIG.
In step S701, it is determined whether a moving image or animation is being reproduced. Whether or not the reproduction is in progress is determined by whether or not the screen is changed without any user operation. If it is determined that playback is in progress, the process advances to step S702; otherwise, the process advances to step S703. In S <b> 702, a value determined in advance for the screen change during reproduction is adopted as the result of the importance calculation. In the case of the example of FIG. 9, the importance is set to 1 in S702.

  In S703, importance calculation processing in the case of other classification described later is performed. The classification is video / animation, but since there is no change specific to the video / animation that affects the importance, the same importance calculation processing as in the other classifications is performed. This process is the same as the process performed in S507 of FIG. 5 and S605 of FIG.

FIG. 8 is a diagram illustrating an example of importance calculation processing in the case of other classifications performed in S507 of FIG. 5, S605 of FIG. 6, and S703 of FIG.
First, in S801, the importance value is initialized to 1. Next, in S802, it is determined whether or not an active window has been switched. If the active window has been switched, the process proceeds to S803, and if not, the process proceeds to S804. In S803, a predetermined value for the switching operation of the active window is added to the importance. In the example of FIG. 9, when the active window is switched, 5 is added to the importance level.

  In step S804, a value calculated from the amount of change in the active window is added to the importance level. In this embodiment, the value is calculated based on the area ratio of the changed portion with respect to the display area of the window. In the example of FIG. 9, a value obtained by dividing the area ratio (%) by the fixed value 10 is added to the importance. Note that the calculation method is not limited to the above-described method. For example, it is possible to make the area ratio and importance non-linear, and it is also possible to calculate based on the number of pixels changed without using the area ratio.

  In S805, the value calculated from the amount of change with respect to the entire screen is added to the importance. In the example of FIG. 9, a value obtained by dividing the area ratio (%) of the changed portion with respect to the entire screen by the fixed value 5 is added to the importance. Note that the calculation method here is not limited to the method of the present embodiment as in the calculation method in S804.

  In step S806, the importance value calculated in steps S801 to S805 is rounded. Since the importance in this embodiment is an integer value of 1 to 10, the importance value is rounded so as to be within the range. The rounding process in this embodiment is performed by rounding off values exceeding 10 to 10 and other values rounded off. In addition, it is good also as rounding up or rounding down instead of rounding off.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (computer program) that implements the functions of the above-described embodiments is supplied to a system or apparatus via a network or various computer-readable storage media. Then, the computer (or CPU, MPU, etc.) of the system or apparatus reads out and executes the program.

201 operation event detection unit, 202 screen change detection unit, 203 importance calculation unit, 204 threshold determination unit, 205 display screen duration measurement unit, 206 image recording unit

Claims (14)

A capture image recording device for recording a display screen,
An operation event detection means for detecting a user operation event;
Screen change detecting means for detecting a change in the display screen;
Based on the operation event detected by the operation event detecting means or the screen change detected by the screen change detecting means, the display screen duration time until the capture image recording means records the captured image of the display screen. Threshold determination means for determining a threshold;
Display screen duration measuring means for measuring the time that the display screen continues without changing, and
Control for controlling the capture image recording means to record the capture image of the display screen when the display screen duration measured by the display screen duration measurement means exceeds the threshold value determined by the threshold value determination means And a captured image recording apparatus.   The captured image recording apparatus according to claim 1, wherein the threshold value determination unit analyzes and classifies the contents of an active window and performs individual threshold value determination processing for each classification.   The threshold value determining means changes the threshold value determining method depending on whether the operation event detected by the operation event detecting means or the screen change detected by the screen change detecting means is for an active window. The captured image recording apparatus according to claim 1.   The captured image recording apparatus according to claim 1, wherein the threshold value determination unit changes a threshold value determination method depending on whether an active window switching operation is performed. The threshold value determining unit includes an importance level calculating unit that calculates the importance level of the screen change based on the operation event detected by the operation event detecting unit or the screen change detected by the screen change detecting unit,
The threshold value determining unit determines a threshold value of a display screen duration time until the capture image recording unit records a capture image of the display screen according to the importance level calculated by the importance level calculating unit. The captured image recording apparatus according to claim 1, wherein the captured image recording apparatus is a recording apparatus.   The captured image recording apparatus according to claim 5, wherein the threshold value determining unit lowers the threshold value as the importance level calculated by the importance level calculating unit increases, and increases the threshold value as the importance level decreases. A captured image recording method for recording a display screen,
An operation event detection process for detecting a user operation event;
A screen change detection step for detecting a change in the display screen;
Based on the operation event detected in the operation event detection step or the screen change detected in the screen change detection step, the display screen duration time until the capture image recording step records the captured image on the display screen. A threshold value determining step for determining a threshold value;
A display screen duration measuring step for measuring the time that the display screen continues without changing, and
Control for controlling the capture image recording step to record a capture image of the display screen when the display screen duration measured in the display screen duration measurement step exceeds the threshold value determined in the threshold value determination step And a captured image recording method.   8. The captured image recording method according to claim 7, wherein in the threshold value determining step, the contents of an active window are analyzed and classified, and an individual threshold value determining process is performed for each classification.   The threshold value determining step changes a threshold value determining method depending on whether the operation event detected in the operation event detecting step or the screen change detected in the screen change detecting step is for an active window. The captured image recording method according to claim 7.   8. The captured image recording method according to claim 7, wherein the threshold value determining step changes the threshold value determining method depending on whether or not an active window switching operation is performed. The threshold value determining step includes an importance level calculating step of calculating the importance level of the screen change based on the operation event detected in the operation event detecting step or the screen change detected in the screen change detecting step,
The threshold value determining step determines a threshold value of a display screen duration time until the capture image recording step records a capture image of the display screen according to the importance level calculated in the importance level calculating step. The captured image recording method according to claim 7, wherein the method is a captured image recording method.   12. The captured image recording method according to claim 11, wherein in the threshold value determining step, the threshold value is lowered as the importance level calculated in the importance level calculating step increases, and the threshold value is increased as the importance level decreases. A program for causing a computer to execute a captured image recording method for recording a display screen,
An operation event detection process for detecting a user operation event;
A screen change detection step for detecting a change in the display screen;
Based on the operation event detected in the operation event detection step or the screen change detected in the screen change detection step, the display screen duration time until the capture image recording step records the captured image on the display screen. A threshold value determining step for determining a threshold value;
A display screen duration measuring step for measuring the time that the display screen continues without changing, and
Control for controlling the capture image recording step to record a capture image of the display screen when the display screen duration measured in the display screen duration measurement step exceeds the threshold value determined in the threshold value determination step A program for causing a computer to execute a process.   A computer-readable storage medium storing the program according to claim 13.

Images