JP2011095979A - Screen recording device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To automatically record image data from a point of time when the setting of image data are matched with recording start conditions until a point of time when it is matched with recording end conditions, without altering the software of an application. <P>SOLUTION: This screen recording device is provided with: a recording condition setting part 3 for accepting the setting of image data matched with recording start conditions, and for accepting the setting of the image data matched with recording end conditions; and a screen image analyzing part 4 for comparing image data whose setting has been accepted by the recording condition setting part 3 with image data acquired by a screen image input part 2, and for detecting a recording start timing and a recording end timing by referring to the result of comparison. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a screen recording apparatus that automatically starts recording image data indicating an image on a screen that changes in accordance with execution of an application, and automatically ends the recording of the image data.

A technology for setting in advance conditions for capturing data on a computer (for example, display data, input history, user name, terminal name, etc.), and capturing the data on the computer when the input condition matches the capturing conditions However, it is disclosed in the following Patent Documents 1 and 2, for example.
In other words, this technique analyzes user input information using a pen or a keyboard, determines whether or not the user input information matches the data capture condition, and if it matches the data capture condition, However, it does not detect the recording start timing of the image data indicating the image on the screen that changes with the execution of the application or the recording end timing of the image data.

  Therefore, even if a technique for determining whether the user input information matches the data capture condition is applied to the screen recording apparatus that records the image data, the recording ends from the time when the recording start condition is met. A screen recording apparatus that automatically records image data cannot be configured until a time point that meets the conditions.

JP-A-11-136606 (paragraph numbers [0039] to [0040]) JP 2003-76405 A (paragraph numbers [0015] to [0016])

Since the conventional screen recording apparatus is configured as described above, even if it can be determined whether or not the user input information matches the data capturing condition, the screen recording apparatus changes as the application is executed. It is impossible to detect the recording start timing of image data indicating the image of the image and the recording end timing of image data. For this reason, there has been a problem that image data cannot be automatically recorded from the time when the recording start condition is met to the time when the recording end condition is met.
When a predetermined image is displayed on the screen, the image data can be automatically recorded by modifying the application software so that the image data indicating the image is started. However, since it is necessary to modify the software, applicable applications are limited, and there is a problem that requires a lot of labor.

  The present invention has been made to solve the above-described problems, and without changing the application software, image data is automatically acquired from the time when the recording start condition is met to the time when the recording end condition is met. It is an object of the present invention to obtain a screen recording apparatus capable of recording data.

  The screen recording apparatus according to the present invention receives a setting of image data that matches the recording start condition as a recording start condition when starting recording of the image data acquired by the image data acquisition means, and As a recording end condition at the time of ending recording, a condition setting unit that accepts the setting of image data that matches the recording end condition, the image data that has been accepted by the condition setting unit, and the image acquired by the image data obtaining unit A timing detection means for comparing the data and referring to the comparison result to detect a recording start timing for starting the recording of the image data and a recording end timing for ending the recording of the image data; Recording start timing by timing detection means The recording of the image data acquired by the image data acquisition means is started from the detected time point, and the recording of the image data is ended when the recording end timing is detected by the timing detection means. .

  According to this invention, as the recording start condition when starting the recording of the image data acquired by the image data acquisition means, the setting of the image data that matches the recording start condition is accepted and the recording of the image data is ended. The condition setting means for receiving the setting of image data that matches the recording end condition, and the image data received by the condition setting means and the image data acquired by the image data acquisition means And a timing detection means for detecting a recording start timing for starting the recording of the image data and a recording end timing for ending the recording of the image data with reference to the comparison result, and the image data recording means detects the timing. Whether the recording start timing is detected by the means Since the recording of the image data acquired by the image data acquisition means is started and the recording of the image data is ended when the recording end timing is detected by the timing detection means, the application software is modified. Thus, there is an effect that the image data can be automatically recorded from the time when the recording start condition is met to the time when the recording end condition is met.

It is a block diagram which shows the screen recording apparatus by Embodiment 1 of this invention. It is a flowchart which shows the processing content (a recording preparation phase, a recording start phase) of the screen recording apparatus by Embodiment 1 of this invention. It is a flowchart which shows the processing content (a recording end phase, an image reproduction phase) of the screen recording apparatus by Embodiment 1 of this invention. FIG. 5 is an explanatory diagram illustrating an example of an image file that matches a recording start condition (an image data data file that matches a recording start condition) and an image file that matches a recording end condition (an image data data file that matches a recording end condition); is there. It is explanatory drawing which shows the structural example of the internal buffer from which the image file which matches recording start conditions, and the image file which matches recording end conditions were read. It is explanatory drawing which has 8-bit R, G, B data for every pixel, and shows the image data of Width (1024 pixels) x Height (768 pixels) in the whole screen. It is explanatory drawing which shows the compression system and compression parameter (resolution, frame rate, compression rate) of image data. It is a block diagram which shows the screen recording apparatus by Embodiment 2 of this invention. It is a flowchart which shows the processing content (a recording preparation phase, a recording start phase) of the screen recording apparatus by Embodiment 2 of this invention. It is a flowchart which shows the processing content (a recording end phase, an image reproduction phase) of the screen recording apparatus by Embodiment 2 of this invention. It is a block diagram which shows the screen recording apparatus by Embodiment 3 of this invention. It is explanatory drawing which shows an example of the color designation system by a recording condition setting part. It is a block diagram which shows the screen recording apparatus by Embodiment 4 of this invention. It is explanatory drawing which shows an example of the area | region designation | designated system by a recording condition setting part. It is a block diagram which shows the screen recording apparatus by Embodiment 5 of this invention. An example of an image file sequence that matches the recording start condition (image data data file sequence that matches the recording start condition) and an image file sequence that matches the recording end condition (image data data file sequence that matches the recording end condition) It is explanatory drawing shown. It is explanatory drawing which shows the structural example of the internal buffer in which the image file sequence which matches recording start conditions, and the image file sequence which matches recording end conditions were read. It is a block diagram which shows the screen recording apparatus by Embodiment 6 of this invention. It is explanatory drawing which shows an example of the compression system and compression parameter (resolution, frame rate, compression rate) corresponding to the load of CPU.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing a screen recording apparatus according to Embodiment 1 of the present invention.
In FIG. 1, an application 1 is software executed by a CPU (not shown) of a screen recording device. For example, an operation monitoring screen for notifying an operation status of a certain device or facility, or an abnormality of a certain device or facility, for example. Software that displays an abnormal display screen or the like on the screen (display) of the screen recording device.
The screen image input unit 2 has a screen capture function, and obtains, via a frame buffer, image data indicating an image on the screen (for example, an operation monitoring screen or an abnormal display screen) that changes as the application 1 is executed. To implement. The screen image input unit 2 constitutes image data acquisition means.

  The recording condition setting unit 3 includes a man-machine interface such as a keyboard and a mouse, for example, and recording when starting recording of the image data acquired by the screen image input unit 2 under the operation of the man-machine interface by the user. A process for accepting image data settings that match the recording start conditions as start conditions and accepting image data settings that match the recording end conditions as recording end conditions when ending the recording of the image data To do. The recording condition setting unit 3 constitutes condition setting means.

The screen image analysis unit 4 analyzes the image data set by the recording condition setting unit 3 and the image data acquired by the screen image input unit 2, and compares the image data with the image data. With reference to the comparison result, a process of detecting a recording start timing for starting the recording of the image data and a recording end timing for ending the recording of the image data is performed.
That is, the screen image analysis unit 4 compares the image data that matches the recording start condition received by the recording condition setting unit 3 and the image data that matches the recording end condition and the image data acquired by the screen image input unit 2. Is calculated as the recording start timing, and the difference between the image data and the image data that matches the recording end condition is determined to be a predetermined value. The process which detects the timing which becomes below this threshold value as a recording end timing is implemented.
The screen image analysis unit 4 constitutes timing detection means.

The screen image recording processing unit 5 starts recording the image data acquired by the screen image input unit 2 from the time when the recording start timing is detected by the screen image analysis unit 4, and the recording end timing is set by the screen image analysis unit 4. When detected, the recording of the image data is terminated.
However, when recording the image data, the screen image recording processing unit 5 compresses the image data with a predetermined compression method and compression parameters (resolution, frame rate, compression rate), and the compressed image data is stored in the image data storage unit. Record in 6.
The image data storage unit 6 is a recording medium such as a hard disk for storing the compressed image data.
The screen image recording processing unit 5 and the image data storage unit 6 constitute image data recording means.

  The image reproduction unit 7 reads the image data compressed by the screen image recording processing unit 5 and stored in the image data storage unit 6, performs an expansion process on the image data, and then displays an image according to the image data. Perform playback on the screen. The image reproducing unit 7 constitutes an image reproducing unit.

  In FIG. 1, each of the screen image input unit 2, the recording condition setting unit 3, the screen image analysis unit 4, the screen image recording processing unit 5 and the image reproduction unit 7 which are components of the screen recording apparatus is dedicated hardware (for example, , A semiconductor integrated circuit on which a CPU is mounted, a one-chip microcomputer, or the like is assumed. However, when the screen recording device is configured by a computer, the screen image input unit 2, recording conditions A program describing the processing contents of the setting unit 3, the screen image analysis unit 4, the screen image recording processing unit 5 and the image reproduction unit 7 is stored in the memory of the computer, and the CPU of the computer is stored in the memory. The program may be executed.

Next, the operation will be described.
In the first embodiment, the operation of the screen recording apparatus will be described in the following four phases.
(1) Phase 1: Image data recording preparation (2) Phase 2: Image data recording start (3) Phase 3: Image data recording end (4) Phase 4: Image reproduction FIG. 2 shows an embodiment of the present invention. 2 is a flowchart showing the processing contents (phase 1 and phase 2) of the screen recording apparatus according to 1;
FIG. 3 is a flowchart showing the processing contents (phase 3 and phase 4) of the screen recording apparatus according to the first embodiment of the present invention.

(1) Phase 1
First, the user operates the man-machine interface of the recording condition setting unit 3 to set a recording start condition for starting image data recording and a recording end condition for ending image data recording (see FIG. Step ST1 of 2).
That is, the user requests the CPU of the screen recording apparatus to execute the application 1 by operating the man-machine interface of the recording condition setting unit 3.
As a result, the CPU executes the application 1 so that the image on the screen changes depending on the status of the device or equipment (for example, if no abnormality occurs in the device or equipment, the screen The operation monitoring screen is displayed on the screen, and when an abnormality occurs in equipment or equipment, the abnormality display screen is displayed on the screen.) When the user operates the man-machine interface of the recording condition setting unit 3, recording is performed. An image on the screen that matches the start condition and an image on the screen that matches the recording end condition are specified.

For example, when the user desires to start recording of image data when an abnormality occurs in a device or facility, an abnormality display screen displayed on the screen as the application 1 is executed is designated.
When the abnormal display screen displayed on the screen is designated in this way, the screen image input unit 2 uses the screen capture function to send image data indicating the abnormal display screen via the frame buffer under the instruction of the recording condition setting unit 3. Therefore, the recording condition setting unit 3 sets the image data (image data indicating an abnormal display screen) acquired by the screen image input unit 2 as the image data that matches the recording start condition.

Further, when the user desires to end the recording of the image data when the abnormality of the device or facility is resolved, the operation monitoring screen displayed on the screen as the application 1 is executed is designated. .
As described above, when the operation monitoring screen displayed on the screen is designated, the screen image input unit 2 uses the screen capture function to transmit the image data indicating the operation monitoring screen via the frame buffer under the instruction of the recording condition setting unit 3. Therefore, the recording condition setting unit 3 sets the image data (image data indicating the operation monitoring screen) acquired by the screen image input unit 2 as the image data that matches the recording end condition.

Here, the image data indicating the abnormal display screen is set as the image data matching the recording start condition, and the image data indicating the operation monitoring screen is set as the image data matching the recording end condition. This is merely an example, and image data of any image may be set as long as the image is displayed on the screen as the application 1 is executed.
Further, here, in order to set the image data that matches the recording start condition and the recording end condition, the CPU executes the application 1 and the screen image input unit 2 indicates the image data displayed on the screen. However, the recording condition setting unit 3 may obtain image data that matches the recording start condition and the recording end condition from the outside.

FIG. 4 shows an example of an image file that matches the recording start condition (image data data file that matches the recording start condition) and an image file that matches the recording end condition (image data data file that matches the recording end condition). It is explanatory drawing.
FIG. 4 shows an example in which five image files that match the recording start condition are set and five image files that match the recording end condition are set. However, the number of settings is not limited to five as long as the number is one or more (the larger the number of settings, the higher the accuracy of detecting an abnormal state or the like).
The number of image files that match the recording start condition and the number of image files that match the recording end condition are not limited to the same number, and may be different.
The format of the image data in the image file is not particularly limited, and may be a compressed format or an uncompressed format. In the example of FIG. 4, RGB format, BMP format, and JPEG format image data are set.

(2) Phase 2
The screen image analysis unit 4 reads image data (image data) that matches the recording start condition accepted by the recording condition setting unit 3 and image data (image data) that matches the recording end condition. Image data is loaded into an internal buffer (not shown) (step ST2).
However, if these image data are compressed, they are decompressed before being loaded into the internal buffer.
If a plurality of image data that matches the recording start condition and a plurality of image data that match the recording end condition are set, all the set image data are loaded into the internal buffer.

FIG. 5 is an explanatory diagram showing a configuration example of an internal buffer in which the image file of FIG. 4 (an image file that matches the recording start condition and an image file that matches the recording end condition) is read.
FIG. 6 shows an outline of image data (image data) included in the image file “START001.RGB” that matches the recording start condition. This image data is R data (8 bits) for each pixel. It has G data (8 bits) and B data (8 bits), and is composed of Width (1024 pixels) × Height (768 pixels) data on the entire screen.

The screen image input unit 2 acquires image data indicating an image on the screen that changes as the application 1 is executed via the frame buffer, and outputs the image data to the screen image analysis unit 4 (step ST3).
The screen image input unit 2 acquires the latest image data. However, the image data may be acquired periodically at a predetermined cycle or at a predetermined time. You may acquire.

When the screen image analysis unit 4 receives the latest image data from the screen image input unit 2, the image indicated by the latest image data and the image data matching the recording start condition previously loaded in the internal buffer are indicated. The image is analyzed and the image data is compared with the image data.
That is, the screen image analysis unit 4 performs, for each pixel, the difference (R, G) between the image data matching the recording start condition loaded in the internal buffer and the latest image data output from the screen image input unit 2. , B), and the differences in all pixels are integrated (step ST4).

However, as shown in FIG. 5, when five image files are loaded into the internal buffer as image files that match the recording start condition, the latest for each image data included in the five image files. The difference from the image data (difference in all pixels) is obtained.
That is, the difference between the image data included in the image file “START001.RGB” and the latest image data, the difference between the image data included in the image file “START002.RGB” and the latest image data, the image file The difference between the image data included in “START003.BMP” and the latest image data, the difference between the image data included in the image file “START004.JPG” and the latest image data, and the image file “START005.JPG” The difference between the image data included in “” and the latest image data is calculated.

When the screen image analysis unit 4 calculates the difference (difference in all pixels) between the image data that matches the recording start condition and the latest image data, the screen image analysis unit 4 compares the difference with a predetermined threshold (step ST5).
If the difference exceeds the predetermined threshold (difference> threshold), the screen image analysis unit 4 returns to step ST3 and repeats the processing of steps ST3 to ST5. If the difference is equal to or smaller than the predetermined threshold (difference ≦ (Threshold), it is determined that the recording start condition is satisfied (ie, it is determined that an abnormality has occurred in the device or facility and the abnormality display screen is displayed on the screen), and the recording start timing is detected.

  Here, as shown in FIG. 5, when five image files are loaded into the internal buffer as image files that match the recording start condition, the image data and the image data included in the five image files. If any of the differences is less than or equal to a predetermined threshold, it is determined that the recording start condition is satisfied, but all of the differences between the image data and the image data included in the five image files are predetermined. When the value is equal to or less than the threshold value, it may be determined that the recording start condition is satisfied.

  Also, here, first, the difference between the image data and the image data included in the five image files is calculated, and then each difference is compared with a predetermined threshold value. (For example, “START001.RGB”), the difference between the image data and the image data is calculated, and the difference is compared with a predetermined threshold. If the difference is equal to or smaller than the predetermined threshold, the recording start condition is set. If it is determined that the difference is greater than a predetermined threshold value, the difference between the image data and the image data included in another image file (for example, “START002.RGB”) is calculated, and the difference is calculated. The difference may be compared with a predetermined threshold value.

When the recording start timing is detected by the screen image analysis unit 4, the screen image recording processing unit 5 starts recording the image data acquired by the screen image input unit 2 from the time when the recording start timing is detected. (Step ST6). If the recording start timing is not detected by the screen image analysis unit 4, recording of the image data acquired by the screen image input unit 2 is not started.
However, when the image data is recorded in the image data storage unit 6, the screen image recording processing unit 5 compresses the image data with a predetermined compression method and compression parameters (resolution, frame rate, compression rate), and performs compression. Image data is recorded in the image data storage unit 6.

FIG. 7 is an explanatory diagram showing a compression method and compression parameters (resolution, frame rate, compression rate) of image data.
FIG. 7 shows an example in which the compression method is JPEG, the resolution is 100%, the frame rate is 10 fps, and the compression rate is 1/20.
The image data compression method need not be limited to JPEG. For example, MPEG-1, MPEG-2, MPEG-4, H.264, and the like. H.264 or the like may be used.

(3) Phase 3
The screen image input unit 2 acquires image data indicating an image on the screen that changes as the application 1 is executed via the frame buffer (step ST11 in FIG. 3).
In phase 3, since image data recording is started, the screen image input unit 2 outputs the image data acquired via the frame buffer to the screen image analysis unit 4 and the screen image recording processing unit 5.

When receiving the latest image data from the screen image input unit 2, the screen image analysis unit 4 indicates the image indicated by the latest image data and the image data that matches the recording end condition previously loaded in the internal buffer. The image is analyzed and the image data is compared with the image data.
That is, the screen image analysis unit 4 calculates, for each pixel, the difference (R, G) between the image data matching the recording end condition loaded in the internal buffer and the latest image data output from the screen image input unit 2. , B), and the differences in all pixels are integrated (step ST12).

However, as shown in FIG. 5, when five image files are loaded into the internal buffer as image files that meet the recording end condition, the latest image data is included in each of the image data included in the five image files. The difference from the image data (difference in all pixels) is obtained.
That is, the difference between the image data included in the image file “STOP001.RGB” and the latest image data, the difference between the image data included in the image file “STOP002.RGB” and the latest image data, the image file The difference between the image data included in “STOP003.BMP” and the latest image data, the difference between the image data included in the image file “STOP004.JPG” and the latest image data, the image file “STOP005.JPG” The difference between the image data included in “” and the latest image data is calculated.

When the screen image analysis unit 4 calculates the difference (difference in all pixels) between the image data matching the recording end condition and the latest image data, the screen image analysis unit 4 compares the difference with a predetermined threshold (step ST13).
If the difference exceeds a predetermined threshold (difference> threshold), the screen image analysis unit 4 determines that the recording end condition is not satisfied (the abnormality of the device or equipment continues, and the abnormality display If it is determined that the screen is displayed on the screen), the recording end timing is not detected.
On the other hand, if the difference is equal to or less than the predetermined threshold (difference ≦ threshold), it is determined that the recording end condition is satisfied (the abnormality of the device or equipment is resolved, and the operation monitoring screen is displayed on the screen. The recording end timing is detected.

  Here, as shown in FIG. 5, when five image files are loaded into the internal buffer as image files that meet the recording end condition, the image data and the image data included in the five image files. If any of the differences is less than or equal to a predetermined threshold, it is determined that the recording end condition is satisfied, but all of the differences between the image data and the image data included in the five image files are predetermined. When the value is equal to or smaller than the threshold value, it may be determined that the recording end condition is satisfied.

  Also, here, first, the difference between the image data and the image data included in the five image files is calculated, and then each difference is compared with a predetermined threshold value. (For example, “STOP001.RGB”), the difference between the image data and the image data is calculated, the difference is compared with a predetermined threshold, and if the difference is equal to or smaller than the predetermined threshold, the recording end condition is set. If it is determined that the difference is greater than a predetermined threshold, the difference between the image data and the image data included in another image file (for example, “STOP002.RGB”) is calculated, and the difference is calculated. The difference may be compared with a predetermined threshold value.

If the screen image analysis unit 4 does not detect the recording end timing, the screen image recording processing unit 5 continues to record the image data acquired by the screen image input unit 2 (step ST14).
On the other hand, when the screen image analysis unit 4 detects the recording end timing, the recording of the image data acquired by the screen image input unit 2 is stopped when the recording end timing is detected (step ST15).

(4) Phase 4
When the screen image recording processing unit 5 stops recording the image data, the image reproducing unit 7 reads the image data compressed and stored in the image data storage unit 6 under the instruction of the user, for example. After the data is decompressed, the image is reproduced on the screen according to the image data (step ST16).

  As is apparent from the above, according to the first embodiment, the image data that matches the recording start condition is set as the recording start condition when starting the recording of the image data acquired by the screen image input unit 2. The recording condition setting unit 3 that receives the setting of image data that matches the recording end condition and the recording condition setting unit 3 receive the setting as the recording end condition when the recording of the image data is ended. The image data and the image data acquired by the screen image input unit 2 are compared, and by referring to the comparison result, the recording start timing for starting the recording of the image data and the recording end timing for ending the recording of the image data are detected. A screen image analysis unit 4 for performing screen image recording processing 5 The recording of the image data acquired by the screen image input unit 2 is started from the time when the recording start timing is detected by the screen image analysis unit 4, and when the recording end timing is detected by the screen image analysis unit 4, Since the recording of the image data is completed, the image data can be automatically recorded from the time when the recording start condition is met to the time when the recording end condition is met without modifying the software of the application 1. There is an effect that can.

Embodiment 2. FIG.
FIG. 8 is a block diagram showing a screen recording apparatus according to Embodiment 2 of the present invention. In the figure, the same reference numerals as those in FIG.
The recording condition setting unit 11 includes a man-machine interface such as a keyboard and a mouse, for example, and recording when starting recording of image data acquired by the screen image input unit 2 under the operation of the man-machine interface by the user. As a start condition, a process of accepting the setting of image data that matches the recording start condition is performed. The recording condition setting unit 11 constitutes a condition setting unit.

The screen image analysis unit 12 analyzes the image data set by the recording condition setting unit 11 and the image data acquired by the screen image input unit 2, thereby comparing the image data with the image data. With reference to the comparison result, a process of detecting a recording start timing for starting the recording of the image data and a recording end timing for ending the recording of the image data is performed.
That is, the screen image analysis unit 12 calculates the difference between the image data that matches the recording start condition accepted by the recording condition setting unit 11 and the image data acquired by the screen image input unit 2, and uses the recording start condition as the recording start condition. The timing at which the difference between the matching image data and the image data exceeds a predetermined threshold is detected as the recording start timing, and the timing at which the difference between the image data and the image data that matches the recording start condition is equal to or less than the predetermined threshold is the recording end timing. The process to detect is performed.
The screen image analysis unit 12 constitutes timing detection means.

  In FIG. 8, each of the screen image input unit 2, the recording condition setting unit 11, the screen image analysis unit 12, the screen image recording processing unit 5 and the image reproduction unit 7 which are components of the screen recording apparatus is dedicated hardware (for example, , A semiconductor integrated circuit on which a CPU is mounted, a one-chip microcomputer, or the like is assumed. However, when the screen recording device is configured by a computer, the screen image input unit 2, recording conditions A program describing the processing contents of the setting unit 11, the screen image analysis unit 12, the screen image recording processing unit 5 and the image reproduction unit 7 is stored in a computer memory, and the CPU of the computer is stored in the memory. The program may be executed.

  In the first embodiment, the recording condition setting unit 3 accepts setting of image data that matches the recording end condition in addition to the image data that matches the recording start condition, and the screen image analysis unit 4 receives the recording condition setting unit 3. For detecting the recording start timing and the recording end timing by analyzing the image data that matches the recording start condition and the recording end condition for which the setting has been received and the image of the image data acquired by the screen image input unit 2 As shown, the recording condition setting unit 11 accepts setting of image data that matches the recording start condition (does not accept setting of image data that matches the recording end condition), and the screen image analysis unit 12 receives the recording condition setting unit 11. The screen data input unit 2 and the image data that matches the recording start conditions accepted by By analyzing the image of a more captured image data, it may be detected recording end timing and the recording start timing.

Next, the operation will be described.
FIG. 9 is a flowchart showing the processing contents (phase 1 and phase 2) of the screen recording apparatus according to the second embodiment of the present invention.
FIG. 10 is a flowchart showing the processing contents (phase 3 and phase 4) of the screen recording apparatus according to the second embodiment of the present invention.

(1) Phase 1
First, the user operates the man-machine interface of the recording condition setting unit 11 to set a recording start condition for starting image data recording (step ST21 in FIG. 9).
That is, the user requests the CPU of the screen recording apparatus to execute the application 1 by operating the man-machine interface of the recording condition setting unit 11.
As a result, the CPU executes the application 1 so that the image on the screen changes depending on the status of the device or equipment (for example, if no abnormality occurs in the device or equipment, the screen The operation monitoring screen is displayed on the screen, and when an abnormality occurs in the equipment or equipment, the abnormality display screen is displayed on the screen.) When the user operates the man-machine interface of the recording condition setting unit 11, recording is performed. Specifies an image on the screen that matches the start condition.

For example, when the user desires to start recording image data when an abnormality occurs in a device or facility, a screen displayed on the screen as the application 1 is executed is an operation monitoring screen ( When switching from the screen when equipment or equipment is normal) to the error display screen, specify the operation monitoring screen that is the normal screen.
As described above, when the operation monitoring screen displayed on the screen is designated, the screen image input unit 2 uses the screen capture function to transmit the image data indicating the operation monitoring screen via the frame buffer under the instruction of the recording condition setting unit 11. Therefore, the recording condition setting unit 11 sets the image data (image data indicating the operation monitoring screen) acquired by the screen image input unit 2 as the image data that matches the recording start condition.
In this second embodiment as well, for the convenience of explanation, a record original acceptance notice was sent as in the first embodiment. Assume that five files are set as image files that match the start conditions (data files of image data that match the recording start conditions) (see FIG. 4).

Here, the image data indicating the operation monitoring screen is set as the image data that matches the recording start condition. However, this is only an example, and the image displayed on the screen as the application 1 is executed. Any image data may be set as long as it is. For example, you may make it set the image data of an abnormal display screen. However, in this case, the determination logic in step ST25 of FIG. 9 and step ST33 of FIG.
Also, here, in order to set image data that matches the recording start condition, the CPU executes the application 1 and the screen image input unit 2 acquires image data indicating an image displayed on the screen. However, the recording condition setting unit 11 may obtain image data that matches the recording start condition from the outside.

(2) Phase 2
The screen image analysis unit 12 reads image data (image data) that matches the recording start conditions accepted by the recording condition setting unit 11, and loads the image data into an internal buffer (not shown) (step ST22).
However, if the image data is compressed, it is decompressed before being loaded into the internal buffer.
In addition, when a plurality of image data matching the recording start condition is set, all the set image data are loaded into the internal buffer.

As in the first embodiment, the screen image input unit 2 acquires image data indicating an image on the screen that changes with the execution of the application 1 via the frame buffer, and the image data is acquired by the screen image analysis unit 12. (Step ST23).
The screen image input unit 2 acquires the latest image data. However, the image data may be acquired periodically at a predetermined cycle or at a predetermined time. You may acquire.

When receiving the latest image data from the screen image input unit 2, the screen image analysis unit 12 loads the image indicated by the latest image data and the internal buffer first, as in the screen image analysis unit 4 of FIG. The image indicated by the image data matching the recording start condition is analyzed, and the image data is compared with the image data.
That is, the screen image analysis unit 12 calculates, for each pixel, the difference (R, G) between the image data matching the recording start condition loaded in the internal buffer and the latest image data output from the screen image input unit 2. , B), and the differences in all pixels are integrated (step ST24).

However, as shown in FIG. 5, when five image files are loaded into the internal buffer as image files that match the recording start condition, the latest for each image data included in the five image files. The difference from the image data (difference in all pixels) is obtained.
That is, the difference between the image data included in the image file “START001.RGB” and the latest image data, the difference between the image data included in the image file “START002.RGB” and the latest image data, the image file The difference between the image data included in “START003.BMP” and the latest image data, the difference between the image data included in the image file “START004.JPG” and the latest image data, and the image file “START005.JPG” The difference between the image data included in “” and the latest image data is calculated.

When the screen image analysis unit 12 calculates a difference (difference in all pixels) between the image data matching the recording start condition and the latest image data, the screen image analysis unit 12 compares the difference with a predetermined threshold (step ST25).
If the difference is equal to or smaller than the predetermined threshold (difference ≦ threshold), the screen image analysis unit 12 returns to step ST23 and repeats the processing of steps ST23 to ST25, but if the difference exceeds the predetermined threshold (difference> (Threshold), judging that the recording start condition is satisfied (determining that an abnormality has occurred in the device or equipment, and that the screen displayed on the screen has been switched from the operation monitoring screen to the abnormality display screen) ) To detect the recording start timing.

  Here, as shown in FIG. 5, when five image files are loaded into the internal buffer as image files that match the recording start condition, the image data and the image data included in the five image files. If any of the differences exceeds a predetermined threshold, it is determined that the recording start condition is satisfied, but all of the differences between the image data and the image data included in the five image files are predetermined. If the threshold is exceeded, it may be determined that the recording start condition is satisfied.

  Also, here, first, the difference between the image data and the image data included in the five image files is calculated, and then each difference is compared with a predetermined threshold value. The difference between the image data and the image data included in (for example, “START001.RGB”) is calculated, the difference is compared with a predetermined threshold, and if the difference exceeds the predetermined threshold, the recording start condition If the difference is equal to or less than a predetermined threshold, the difference between the image data and the image data included in another image file (for example, “START002.RGB”) is calculated, The difference may be compared with a predetermined threshold value.

When the recording start timing is detected by the screen image analysis unit 12, the screen image recording processing unit 5 is acquired by the screen image input unit 2 from the time when the recording start timing is detected, as in the first embodiment. Recording of the processed image data is started (step ST26). If the recording start timing is not detected by the screen image analysis unit 12, the recording of the image data acquired by the screen image input unit 2 is not started.
However, when the image data is recorded in the image data storage unit 6, the screen image recording processing unit 5 compresses the image data with a predetermined compression method and compression parameters (resolution, frame rate, compression rate), and performs compression. Image data is recorded in the image data storage unit 6.

(3) Phase 3
The screen image input unit 2 acquires image data indicating an image on the screen that changes with the execution of the application 1 via the frame buffer (step ST31 in FIG. 10).
In phase 3, since image data recording is started, the screen image input unit 2 outputs the image data acquired via the frame buffer to the screen image analysis unit 12 and the screen image recording processing unit 5.

When the latest image data is received from the screen image input unit 2, the screen image analysis unit 12 indicates the image indicated by the latest image data and the image data that matches the recording start condition previously loaded in the internal buffer. The image is analyzed and the image data is compared with the image data.
That is, the screen image analysis unit 12 calculates, for each pixel, the difference (R, G) between the image data matching the recording start condition loaded in the internal buffer and the latest image data output from the screen image input unit 2. , B), and the differences in all the pixels are integrated (step ST32).

However, as shown in FIG. 5, when five image files are loaded into the internal buffer as image files that match the recording start condition, the latest for each image data included in the five image files. The difference from the image data (difference in all pixels) is obtained.
That is, the difference between the image data included in the image file “START001.RGB” and the latest image data, the difference between the image data included in the image file “START002.RGB” and the latest image data, the image file The difference between the image data included in “START003.BMP” and the latest image data, the difference between the image data included in the image file “START004.JPG” and the latest image data, and the image file “START005.JPG” The difference between the image data included in “” and the latest image data is calculated.

When the screen image analysis unit 12 calculates the difference (difference in all pixels) between the image data matching the recording start condition and the latest image data, the screen image analysis unit 12 compares the difference with a predetermined threshold (step ST33).
If the difference exceeds a predetermined threshold (difference> threshold), the screen image analysis unit 12 determines that the recording start condition is satisfied as before (the recording end condition is not satisfied) (device or It is determined that the abnormality of the equipment or the like continues and the abnormality display screen is displayed on the screen), and the recording end timing is not detected.
On the other hand, if the difference is equal to or less than the predetermined threshold (difference ≦ threshold), it is determined that the recording start condition is no longer satisfied (the recording end condition is satisfied) (the abnormality of the device or equipment is resolved) Then, it is determined that the screen displayed on the screen is switched from the abnormality display screen to the operation monitoring screen), and the recording end timing is detected.

  Here, as shown in FIG. 5, when five image files are loaded into the internal buffer as image files that match the recording start condition, the image data and the image data included in the five image files. If any of the differences is less than or equal to a predetermined threshold, it is determined that the recording end condition is satisfied, but all of the differences between the image data and the image data included in the five image files are predetermined. When the value is equal to or smaller than the threshold value, it may be determined that the recording end condition is satisfied.

  Also, here, first, the difference between the image data and the image data included in the five image files is calculated, and then each difference is compared with a predetermined threshold value. (For example, “START001.RGB”), the difference between the image data and the image data is calculated, the difference is compared with a predetermined threshold, and if the difference is equal to or smaller than the predetermined threshold, the recording end condition is set. If it is determined that the difference is greater than a predetermined threshold value, the difference between the image data and the image data included in another image file (for example, “START002.RGB”) is calculated, and the difference is calculated. The difference may be compared with a predetermined threshold value.

As in the first embodiment, the screen image recording processing unit 5 continues to record the image data acquired by the screen image input unit 2 unless the screen image analysis unit 12 detects the recording end timing (step 1). ST34).
On the other hand, when the screen image analysis unit 12 detects the recording end timing, the recording of the image data acquired by the screen image input unit 2 is stopped when the recording end timing is detected (step ST35).

(4) Phase 4
When the screen image recording processing unit 5 stops recording the image data, the image reproducing unit 7 reads the image data compressed and stored in the image data storage unit 6 under the instruction of the user, for example. After the data is expanded, the image is reproduced on the screen according to the image data (step ST36).

  As is apparent from the above, according to the second embodiment, as the recording start condition when starting the recording of the image data acquired by the screen image input unit 2, setting of image data that matches the recording start condition is set. The recording condition setting unit 11 that receives the image data, the image data received by the recording condition setting unit 11 and the image data acquired by the screen image input unit 2 are compared, and the comparison result is referred to to compare the image data A screen image analysis unit 12 for detecting a recording start timing for starting recording and a recording end timing for ending recording of image data. The screen image analysis processing unit 5 detects the recording start timing by the screen image analysis unit 12. Acquired by the screen image input unit 2 Since the recording of the image data is started and the recording of the image data is terminated when the recording end timing is detected by the screen image analysis unit 12, the recording is started without modifying the software of the application 1. There is an effect that the image data can be automatically recorded from the time point meeting the condition to the time point meeting the recording end condition.

Embodiment 3 FIG.
FIG. 11 is a block diagram showing a screen recording apparatus according to Embodiment 3 of the present invention. In the figure, the same reference numerals as those in FIG.
As with the recording condition setting unit 3 in FIG. 1, the recording condition setting unit 13 includes a man-machine interface such as a keyboard and a mouse, for example. Under the operation of the man-machine interface by the user, the screen image input unit 2 As a recording start condition when starting the recording of the image data acquired by the above, the setting of image data that matches the recording start condition is accepted, and as a recording end condition when the recording of the image data is ended, the recording Processing for accepting the setting of image data that matches the end condition is performed.
However, unlike the recording condition setting unit 3 in FIG. 1, the recording condition setting unit 13 has a function of accepting the setting of the color of an area to be excluded from the comparison target in the image on the screen.
The recording condition setting unit 13 constitutes a condition setting unit.

Similar to the screen image analysis unit 4 in FIG. 1, the screen image analysis unit 14 analyzes the image data received by the recording condition setting unit 13 and the image data acquired by the screen image input unit 2. Then, the image data is compared with the image data, and a process of detecting a recording start timing for starting the recording of the image data and a recording end timing for ending the recording of the image data is performed by referring to the comparison result.
That is, the screen image analysis unit 14 determines the difference between the image data matching the recording start condition and the image data matching the recording end condition received by the recording condition setting unit 13 and the image data acquired by the screen image input unit 2. Is calculated as the recording start timing, and the difference between the image data and the image data that matches the recording end condition is determined to be a predetermined value. The process which detects the timing which becomes below this threshold value as a recording end timing is implemented.
However, unlike the screen image analysis unit 4 in FIG. 1, the screen image analysis unit 14 has the same color as the color accepted by the recording condition setting unit 13 among the image data acquired by the screen image input unit 2. The image data in the area is excluded from the comparison target, and the remaining image data not excluded from the comparison target is compared with the image data that matches the recording start condition and the recording end condition accepted by the recording condition setting unit 13.
Note that the screen image analysis unit 14 constitutes timing detection means.

  In FIG. 11, each of the screen image input unit 2, the recording condition setting unit 13, the screen image analysis unit 14, the screen image recording processing unit 5 and the image reproduction unit 7 which are components of the screen recording apparatus is dedicated hardware (for example, , A semiconductor integrated circuit on which a CPU is mounted, a one-chip microcomputer, or the like is assumed. However, when the screen recording device is configured by a computer, the screen image input unit 2, recording conditions A program describing the processing contents of the setting unit 13, the screen image analysis unit 14, the screen image recording processing unit 5 and the image reproduction unit 7 is stored in the memory of a computer, and the CPU of the computer is stored in the memory. The program may be executed.

Next, the operation will be described.
However, as compared with the recording condition setting unit 3 and the screen image analysis unit 4 in the first embodiment, the recording condition setting unit 13 has a function of accepting the setting of the color of the area excluded from the comparison target, and the screen image analysis unit 14 is the same except that the image data of the same color area as the color accepted by the recording condition setting unit 13 is excluded from the comparison target. Therefore, the recording condition setting unit 13 and the screen image analysis unit 14 are the same. Only the processing contents of will be described.

As in the recording condition setting unit 3 in FIG. 1, the recording condition setting unit 13 accepts setting of image data that matches the recording start condition as a recording start condition when recording image data is started in phase 1. At the same time, as a recording end condition at the time of ending the recording of the image data, a process of accepting setting of image data that matches the recording end condition is performed.
In addition, the recording condition setting unit 13 performs a process of accepting the setting of the color (for example, background color) of the area to be excluded from the comparison target in the image on the screen.

For example, when the CPU of the screen recording apparatus executes the application 1 and an image is displayed on the screen, the user can change the man-machine interface of the recording condition setting unit 13 as shown in FIG. When an area (for example, a background area) to be excluded from the comparison target is specified by operation (in the example of FIG. 12A, the background area is specified with the mouse pointer), the recording condition setting unit 13 causes the area to be The color held by the pixel is set as a color that is not subject to image analysis (the color of the area that is excluded from the comparison target).
Alternatively, the recording condition setting unit 13 analyzes the colors constituting the image displayed on the screen, and a list of colors constituting the image is displayed as shown in FIG. Display, and the user operates the man-machine interface of the recording condition setting unit 13 to select the color of the area to be excluded from the comparison target from the list, and set that color as the image analysis target designated color To do. In the example of FIG. 12B, red is set as a non-image analysis target designated color.
Note that the number of designated non-image analysis target colors may be one, but a plurality may be set as necessary.

In the phase 2, the screen image analysis unit 14 obtains the image data that matches the recording start condition received by the recording condition setting unit 13 and the screen image input unit 2 in the same manner as the screen image analysis unit 4 in FIG. A process of calculating a difference between the obtained image data and detecting a timing at which the difference between the image data and the image data matching the recording start condition becomes equal to or less than a predetermined threshold is performed as a recording start timing.
However, unlike the screen image analysis unit 4 in FIG. 1, the screen image analysis unit 14 matches the image start conditions received by the recording condition setting unit 13 and the image acquired by the screen image input unit 2. When calculating the data difference, among the image data acquired by the screen image input unit 2, the image data of the region of the same color as the designated color not to be analyzed that has been set by the recording condition setting unit 13 is calculated. Exclude from calculation.

That is, the screen image analysis unit 14, similar to the screen image analysis unit 4 in FIG. 1, calculates, for each pixel, the image data that matches the recording start condition and the latest image data output from the screen image input unit 2. The difference (difference for each of R, G, and B) is calculated and the difference in all the pixels is integrated. Among the pixels constituting the image data acquired by the screen image input unit 2, the image analysis target is not specified. For pixels holding the same color as the color, the difference is not calculated and not included in the integration.
In the example of FIG. 12, since the color of the background area is set to a non-image analysis target designated color in the image displayed on the screen, the area excluding the background area (menu 1, button 1, button 2). , Button 3, window 1, window 2, window 3) and the difference is calculated and integrated.

In the phase 3, the screen image analysis unit 14 acquires the image data that matches the recording end condition received by the recording condition setting unit 13 and the screen image input unit 2 in the same manner as the screen image analysis unit 4 of FIG. 1. A process of calculating a difference between the image data and detecting a timing at which a difference between the image data and the image data matching the recording end condition is equal to or less than a predetermined threshold is performed as a recording end timing.
However, unlike the screen image analysis unit 4 of FIG. 1, the screen image analysis unit 14 has image data that matches the recording end condition accepted by the recording condition setting unit 13 and the image acquired by the screen image input unit 2. When calculating the data difference, among the image data acquired by the screen image input unit 2, the image data of the region of the same color as the designated color not to be analyzed that has been set by the recording condition setting unit 13 is calculated. Exclude from calculation.

That is, the screen image analysis unit 14, similar to the screen image analysis unit 4 of FIG. 1, calculates, for each pixel, the image data that matches the recording end condition and the latest image data output from the screen image input unit 2. The difference (difference for each of R, G, and B) is calculated and the difference in all the pixels is integrated. Among the pixels constituting the image data acquired by the screen image input unit 2, the image analysis target is not specified. For pixels holding the same color as the color, the difference is not calculated and not included in the integration.
In the example of FIG. 12, since the color of the background area is set to a non-image analysis target designated color in the image displayed on the screen, the area excluding the background area (menu 1, button 1, button 2). , Button 3, window 1, window 2, window 3) and the difference is calculated and integrated.

  As is apparent from the above, according to the third embodiment, the recording condition setting unit 13 has a function of accepting the setting of the color of the area to be excluded from the comparison target in the image on the screen, and the screen image analysis unit 14 However, among the image data acquired by the screen image input unit 2, the image data of the same color as the color accepted by the recording condition setting unit 13 is excluded from the comparison target and is not excluded from the comparison target. Since the remaining image data is compared with the image data that matches the recording start condition and the recording end condition received by the recording condition setting unit 13, the recording start timing and recording are compared with those in the first embodiment. The amount of processing when detecting the end timing is reduced, and the recording start timing and recording end timing are detected. An effect that can increase the speed.

  In the third embodiment, the recording condition setting unit 13 and the screen image analysis unit 14 are mounted instead of the recording condition setting unit 3 and the screen image analysis unit 4 in FIG. And a function for excluding image data in the same color area as the color accepted by the recording condition setting unit 13 from the comparison target are added to the screen recording apparatus of the first embodiment. Although shown, the recording condition setting unit 13 and the screen image analysis unit 14 are mounted instead of the recording condition setting unit 11 and the screen image analysis unit 12 in FIG. A function for accepting settings and image data in the same color area as the color accepted by the recording condition setting unit 13 are excluded from comparison targets. That function and may be added to the screen printing apparatus of the second embodiment of.

Embodiment 4 FIG.
FIG. 13 is a block diagram showing a screen recording apparatus according to Embodiment 4 of the present invention. In the figure, the same reference numerals as those in FIG.
The recording condition setting unit 15 includes, for example, a man-machine interface such as a keyboard and a mouse, like the recording condition setting unit 3 in FIG. As a recording start condition when starting the recording of the image data acquired by the above, the setting of image data that matches the recording start condition is accepted, and as a recording end condition when the recording of the image data is ended, the recording Processing for accepting the setting of image data that matches the end condition is performed.
However, unlike the recording condition setting unit 3 in FIG. 1, the recording condition setting unit 15 has a function of accepting setting of a comparison target region in an image on the screen.
The recording condition setting unit 15 constitutes a condition setting unit.

The screen image analysis unit 16 analyzes the image data set by the recording condition setting unit 15 and the image data acquired by the screen image input unit 2 in the same manner as the screen image analysis unit 4 of FIG. Then, the image data is compared with the image data, and a process of detecting a recording start timing for starting the recording of the image data and a recording end timing for ending the recording of the image data is performed by referring to the comparison result.
That is, the screen image analysis unit 16 determines the difference between the image data that matches the recording start condition received by the recording condition setting unit 15 and the image data that matches the recording end condition and the image data acquired by the screen image input unit 2. Is calculated as the recording start timing, and the difference between the image data and the image data that matches the recording end condition is determined to be a predetermined value. The process which detects the timing which becomes below this threshold value as a recording end timing is implemented.
However, unlike the screen image analysis unit 4 in FIG. 1, the screen image analysis unit 16 is image data of an area whose setting is received by the recording condition setting unit 15 from among the image data acquired by the screen image input unit 2. And the image data and the image data matching the recording start condition and the recording end condition accepted by the recording condition setting unit 15 are compared.
The screen image analysis unit 16 constitutes timing detection means.

  In FIG. 13, each of the screen image input unit 2, the recording condition setting unit 15, the screen image analysis unit 16, the screen image recording processing unit 5 and the image reproduction unit 7 which are components of the screen recording device is dedicated hardware (for example, , A semiconductor integrated circuit on which a CPU is mounted, a one-chip microcomputer, or the like is assumed. However, when the screen recording device is configured by a computer, the screen image input unit 2, recording conditions A program describing the processing contents of the setting unit 15, the screen image analysis unit 16, the screen image recording processing unit 5 and the image reproduction unit 7 is stored in the memory of a computer, and the CPU of the computer is stored in the memory. The program may be executed.

Next, the operation will be described.
However, compared with the recording condition setting unit 3 and the screen image analysis unit 4 in the first embodiment, the recording condition setting unit 15 has a function of accepting the setting of the comparison target area, and the screen image analysis unit 16 has the recording condition. Since the process is the same except that the image data of the area for which the setting is accepted by the setting unit 15 is to be compared, only the processing contents of the recording condition setting unit 15 and the screen image analysis unit 16 will be described.

In the phase 1, the recording condition setting unit 15 accepts the setting of image data that matches the recording start condition as the recording start condition when starting the recording of the image data in the same manner as the recording condition setting unit 3 in FIG. At the same time, as a recording end condition at the time of ending the recording of the image data, a process of accepting setting of image data that matches the recording end condition is performed.
In addition, the recording condition setting unit 15 performs a process of accepting the setting of the comparison target area in the image on the screen.

For example, when the CPU of the screen recording apparatus executes the application 1 and an image is displayed on the screen, the user can change the man-machine interface of the recording condition setting unit 15 as shown in FIG. By operating, an arbitrary window is selected from a plurality of windows existing in the image, and an area constituting the window is set as a comparison target area.
In the example of FIG. 14A, the area constituting the window 2 is set as the comparison target area.
In this case, the shape of the area that can be specified depends on the shape of each window existing in the image.

Alternatively, the user operates the man-machine interface of the recording condition setting unit 15 to draw the comparison target region directly on the image on the screen as shown in FIG. Set to the target area.
In the example of FIG. 14B, rectangular areas surrounding window 1, window 2, and window 3 are set as comparison target areas.
In this case, since the shape of the area that can be specified is determined by the user's drawing, the shape is not limited to a rectangular shape, and a free shape such as a polygon can be specified in addition to a basic figure such as an ellipse or a triangle.
Note that only one area may be set for comparison, but a plurality of areas may be set as necessary.

In the phase 2, the screen image analysis unit 16 acquires image data that matches the recording start condition received by the recording condition setting unit 15 and the screen image input unit 2, as in the screen image analysis unit 4 of FIG. 1. A process of calculating a difference between the obtained image data and detecting a timing at which the difference between the image data and the image data matching the recording start condition becomes equal to or less than a predetermined threshold is performed as a recording start timing.
However, unlike the screen image analysis unit 4 in FIG. 1, the screen image analysis unit 16 has image data that matches the recording start condition accepted by the recording condition setting unit 15 and the image acquired by the screen image input unit 2. When calculating the data difference, the image data of the area whose setting is accepted by the recording condition setting unit 15 is extracted from the image data acquired by the screen image input unit 2, and the image data and the recording condition setting unit are extracted. The image data that matches the recording start condition for which the setting has been accepted by 15 is compared.

That is, the screen image analysis unit 16, similar to the screen image analysis unit 4 in FIG. 1, calculates, for each pixel, the image data that matches the recording start condition and the latest image data output from the screen image input unit 2. The difference (difference for each of R, G, and B) is calculated and the differences in all the pixels are integrated. Among the pixels constituting the image data acquired by the screen image input unit 2, the recording condition setting unit 15 For pixels other than the region for which the setting has been accepted, the difference is not calculated and not included in the integration, and only the pixel in the comparison target region is calculated and included in the integration.
In the example of FIG. 12A, since the window 2 is set as a comparison target region in the image displayed on the screen, the difference is calculated and integrated for the pixels constituting the window 2. Is done.
In the example of FIG. 12B, the area surrounding window 1, window 2, and window 3 is set as the comparison target area in the image displayed on the screen. Differences are calculated and integrated for the pixels constituting the area surrounding window 2 and window 3.

In the phase 3, the screen image analysis unit 16 acquires the image data that matches the recording end condition received by the recording condition setting unit 15 and the screen image input unit 2 in the same manner as the screen image analysis unit 4 of FIG. 1. A process of calculating a difference between the obtained image data and detecting a timing at which the difference between the image data and the image data matching the recording end condition becomes equal to or less than a predetermined threshold is performed as a recording start timing.
However, unlike the screen image analysis unit 4 of FIG. 1, the screen image analysis unit 16 has image data that matches the recording end condition accepted by the recording condition setting unit 15 and the image acquired by the screen image input unit 2. When calculating the data difference, the image data of the area whose setting is accepted by the recording condition setting unit 15 is extracted from the image data acquired by the screen image input unit 2, and the image data and the recording condition setting unit are extracted. The image data that matches the recording end condition for which the setting has been accepted in step 15 is compared.

That is, the screen image analysis unit 16, similar to the screen image analysis unit 4 in FIG. 1, calculates, for each pixel, image data that matches the recording end condition and the latest image data output from the screen image input unit 2. The difference (difference for each of R, G, and B) is calculated and the differences in all the pixels are integrated. Among the pixels constituting the image data acquired by the screen image input unit 2, the recording condition setting unit 15 For pixels other than the region for which the setting has been accepted, the difference is not calculated and not included in the integration, and only the pixel in the comparison target region is calculated and included in the integration.
In the example of FIG. 12A, since the window 2 is set as a comparison target region in the image displayed on the screen, the difference is calculated and integrated for the pixels constituting the window 2. Is done.
In the example of FIG. 12B, the area surrounding window 1, window 2, and window 3 is set as the comparison target area in the image displayed on the screen. Differences are calculated and integrated for the pixels constituting the area surrounding window 2 and window 3.

  As apparent from the above, according to the fourth embodiment, the recording condition setting unit 15 has a function of accepting the setting of the comparison target area in the image on the screen, and the screen image analysis unit 16 inputs the screen image. From the image data acquired by the unit 2, image data of an area for which the setting is received by the recording condition setting unit 15 is extracted, and the recording start condition for which the setting is received by the image data and the recording condition setting unit 15, and Since the image data matching the recording end condition is compared, the processing amount when detecting the recording start timing and the recording end timing is reduced as compared with the first embodiment, and the recording start timing is reduced. In addition, the recording end timing detection speed can be increased.

  In the fourth embodiment, instead of the recording condition setting unit 3 and the screen image analysis unit 4 shown in FIG. 1, a recording condition setting unit 15 and a screen image analysis unit 16 are mounted to set the comparison target area. 8 and the function for comparing the image data of the area for which the setting is received by the recording condition setting unit 15 are added to the screen recording apparatus of the first embodiment. A recording condition setting unit 15 and a screen image analysis unit 16 are mounted in place of the condition setting unit 11 and the screen image analysis unit 12, so that the setting of the comparison target area is set and the setting is performed by the recording condition setting unit 15. The screen recording apparatus of the second embodiment is added with a function for comparing the image data of the received area as a comparison target. It may be.

Embodiment 5 FIG.
FIG. 15 is a block diagram showing a screen recording apparatus according to Embodiment 5 of the present invention. In the figure, the same reference numerals as those in FIG.
As with the recording condition setting unit 3 in FIG. 1, the recording condition setting unit 21 includes a man-machine interface such as a keyboard and a mouse, for example. Under the operation of the man-machine interface by the user, the screen image input unit 2 As a recording start condition when starting recording of the image data acquired by the above, the setting of an image data string (a series of image data) that matches the recording start condition is accepted, and an image data string that matches the recording end condition ( A process of accepting the setting of a series of image data) is performed.
The recording condition setting unit 21 constitutes a condition setting unit.

The screen image analysis unit 22 is a moving image of the image data sequence received by the recording condition setting unit 21 and the image data sequence acquired by the screen image input unit 2 (a series of image data acquired by the screen image input unit 2). By analyzing the image, the image data sequence is compared with the image data sequence, and the comparison result is referred to, and the recording start timing for starting the recording of the image data and the recording end timing for ending the recording of the image data are determined. Perform the detection process.
That is, the screen image analysis unit 22 has an image data sequence that matches the recording start condition received by the recording condition setting unit 21, an image data sequence that matches the recording end condition, and the image data acquired by the screen image input unit 2. The difference between the columns is calculated, the timing at which the difference between the image data sequence that matches the recording start condition and the image data sequence falls below a predetermined threshold is detected as the recording start timing, and the image data sequence that matches the recording end condition A process of detecting a timing at which the difference between the image data strings becomes equal to or less than a predetermined threshold as a recording end timing is performed.
The screen image analysis unit 22 constitutes timing detection means.

  In FIG. 15, each of the screen image input unit 2, the recording condition setting unit 21, the screen image analysis unit 22, the screen image recording processing unit 5, and the image reproduction unit 7 which are components of the screen recording device is dedicated hardware (for example, , A semiconductor integrated circuit on which a CPU is mounted, a one-chip microcomputer, or the like is assumed. However, when the screen recording device is configured by a computer, the screen image input unit 2, recording conditions A program describing the processing contents of the setting unit 21, the screen image analysis unit 22, the screen image recording processing unit 5, and the image reproduction unit 7 is stored in a computer memory, and the CPU of the computer is stored in the memory. The program may be executed.

  In the first embodiment, the recording condition setting unit 3 receives the setting of the image data that matches the recording start condition and the recording end condition, and the screen image analysis unit 4 receives the image data that matches the recording start condition and the recording end condition. Although the image data acquired by the screen image input unit 2 is compared (still image analysis), the recording condition setting unit 21 matches the recording start condition and the recording end condition (a series of image data). The screen image analysis unit 22 compares the image data sequence that matches the recording start condition and the recording end condition with the image data sequence (a series of image data) acquired by the screen image input unit 2 (moving image analysis). ).

The processing contents will be specifically described below.
As in the recording condition setting unit 3 in FIG. 1, the recording condition setting unit 21 accepts setting of image data that matches the recording start condition as a recording start condition when recording image data is started in phase 1. At the same time, as a recording end condition at the time of ending the recording of the image data, a process of accepting setting of image data that matches the recording end condition is performed.
However, unlike the recording condition setting unit 3 in FIG. 1, the recording condition setting unit 21 accepts a plurality of settings for image data that matches the recording start condition and the recording end condition.
A plurality of image data (a series of image data) is set as an image data string that matches the recording start condition, and a plurality of image data (a series of image data) is set as an image data string that matches the recording end condition. .

FIG. 16 shows an image file sequence that matches the recording start condition (a data file sequence of image data that matches the recording start condition) and an image file sequence that matches the recording end condition (a data file sequence of image data that matches the recording end condition). It is explanatory drawing which shows an example.
FIG. 16A shows an example in which five image file sequences that match the recording start condition are set, and each of the five image file sequences is composed of three data file sequences of image data. Yes.
In FIG. 16B, five image file sequences that match the recording end condition are set, and each of the five image file sequences includes a data file sequence of three image data. Show.
Here, an example is shown in which five image file examples that match the recording start condition and the recording end condition are set. However, the number is not limited to five as long as the number is set to one or more.
Each image file sequence is composed of three image data data file sequences. However, it is sufficient that each image file sequence is composed of two or more image data data file sequences. It is not limited to the data file sequence of the image data.

In the phase 2, the screen image analysis unit 22 receives an image data sequence (image data sequence) that matches the recording start condition received by the recording condition setting unit 21 and an image data sequence (image data) that matches the recording end condition. The image data strings are loaded into an internal buffer (not shown).
However, when these image data strings are compressed, they are decompressed and loaded into the internal buffer.
If a plurality of image data strings that match the recording start condition and a plurality of image data strings that match the recording end condition are set, all the set image data strings are loaded into the internal buffer.

  FIG. 17 is an explanatory diagram showing a configuration example of an internal buffer in which the image file sequence of FIG. 16 (an image file sequence that matches the recording start condition and an image file sequence that matches the recording end condition) is read.

The screen image input unit 2 acquires, via a frame buffer, an image data sequence (for example, three consecutive image data) indicating an image on the screen that changes as the application 1 is executed, and the image data sequence. Is output to the screen image analysis unit 22.
The screen image input unit 2 acquires the latest image data sequence. However, the acquisition of the image data sequence may be acquired periodically at a predetermined cycle or may be determined in advance. It may be acquired at a time.

When the screen image analysis unit 22 receives the latest image data sequence from the screen image input unit 2, the screen image analysis unit 22 matches the moving image indicated by the latest image data sequence and the recording start condition previously loaded in the internal buffer. The moving image indicated by the data string is analyzed, and the image data string is compared with the image data string.
That is, the screen image analysis unit 22 calculates the difference (difference for each of R, G, and B) for each pixel and accumulates the differences for all the pixels. For example, the recording start condition loaded in the internal buffer When the image file of the image data string that matches “START001-0.RGB”, “START001-1.RGB”, “START001-2.RGB”, first, the image file “START001-0.RGB” The difference between the image data included in the image data and the first image data in the latest image data sequence (three consecutive image data) is calculated.
Next, a difference (difference in all pixels) between the image data included in the image file “START001-1.RGB” and the second image data in the latest image data sequence is calculated, and finally, A difference (difference in all pixels) between the image data included in the image file “START001-2.RGB” and the third image data in the latest image data sequence is calculated.

When the screen image analysis unit 22 calculates the difference (difference in all pixels) between the three pieces of image data and the three pieces of image data, the screen image analysis unit 22 adds (or averages) the differences, and adds (or averages) the differences. Is the final difference value.
However, as shown in FIG. 17A, when five image file sequences are loaded in the internal buffer as image file sequences that match the recording start condition, they are included in the five image file sequences. For each image data string, a difference (difference in all pixels) from the latest image data string is obtained, and each difference is integrated (or averaged).

When obtaining the final difference, the screen image analysis unit 22 compares the final difference with a predetermined threshold value.
Similar to the screen image analysis unit 4 in FIG. 1, the screen image analysis unit 22 determines that the recording start condition is satisfied if the difference is equal to or less than a predetermined threshold (difference ≦ threshold) (equipment or facility). For example, the abnormality display screen is displayed on the screen), and the recording start timing is detected.

  Here, as shown in FIG. 17A, when five image file sequences are loaded in the internal buffer as image file sequences that match the recording start condition, they are included in the five image file sequences. If any of the differences between the image data sequence and the image data sequence is equal to or less than a predetermined threshold value, it is determined that the recording start condition is satisfied, but the image data sequences included in the five image file sequences When all the differences between the image data string and the image data string are equal to or less than a predetermined threshold value, it may be determined that the recording start condition is satisfied.

  Also, here, first, the difference between the image data sequence and the image data sequence included in the five image file sequences is calculated, and then each difference is compared with a predetermined threshold value. The difference between the image data included in the image file sequence (for example, “START001-0.RGB”, “START001-1.RGB”, “START001-2.RGB”) and the image data sequence is calculated. The difference is compared with a predetermined threshold, and if the difference is equal to or less than the predetermined threshold, it is determined that the recording start condition is satisfied. If the difference exceeds the predetermined threshold, another image file sequence ( For example, the difference between the image data included in “START002-0.RGB”, “START002-1.RGB”, “START002-2.RGB”) and the image data string Calculates a may be compared with the difference with a predetermined threshold value.

  Next, in phase 3, the screen image input unit 2 sends an image data string (for example, three consecutive image data) indicating an image on the screen that changes as the application 1 is executed via the frame buffer. The image data sequence is acquired and output to the screen image analysis unit 22.

When receiving the latest image data sequence from the screen image input unit 2 in phase 3, the screen image analysis unit 22 receives the moving image indicated by the latest image data sequence and the recording end condition previously loaded in the internal buffer. The moving image indicated by the image data sequence matching the above is analyzed, and the image data sequence is compared with the image data sequence.
That is, the screen image analysis unit 22 calculates the difference (difference for each R, G, and B) for each pixel and adds up the differences for all the pixels. For example, the recording end condition loaded in the internal buffer When the image file of the image data sequence that matches the above is “STOP001-0.RGB”, “STOP001-1.RGB”, “STOP001-2.RGB”, first, the image file “STOP001-0.RGB”. The difference between the image data included in the image data and the first image data in the latest image data sequence (three consecutive image data) is calculated.
Next, a difference (difference in all pixels) between the image data included in the image file “STOP001-1.RGB” and the second image data in the latest image data sequence is calculated, and finally, A difference (difference in all pixels) between the image data included in the image file “STOP001-2.RGB” and the third image data in the latest image data sequence is calculated.

When the screen image analysis unit 22 calculates the difference (difference in all pixels) between the three pieces of image data and the three pieces of image data, the screen image analysis unit 22 adds (or averages) the differences, and adds (or averages) the differences. Is the final difference value.
However, as shown in FIG. 17B, when five image file sequences are loaded in the internal buffer as image file sequences that match the recording end condition, they are included in the five image file sequences. For each image data string, a difference (difference in all pixels) from the latest image data string is obtained, and each difference is integrated (or averaged).

When obtaining the final difference, the screen image analysis unit 22 compares the final difference with a predetermined threshold value.
Similar to the screen image analysis unit 4 in FIG. 1, the screen image analysis unit 22 determines that the recording end condition is satisfied if the difference is equal to or smaller than a predetermined threshold (difference ≦ threshold) (equipment or facility). And the operation monitoring screen is displayed on the screen), the recording end timing is detected.

  Here, as shown in FIG. 17B, when five image file sequences are loaded in the internal buffer as image file sequences that match the recording end condition, they are included in the five image file sequences. If any of the difference between the image data sequence and the image data sequence is equal to or less than a predetermined threshold, it is determined that the recording end condition is satisfied, but the image data sequences included in the five image file sequences When all the differences between the image data string and the image data string are equal to or less than a predetermined threshold value, it may be determined that the recording end condition is satisfied.

  Also, here, first, the difference between the image data sequence and the image data sequence included in the five image file sequences is calculated, and then each difference is compared with a predetermined threshold value. The difference between the image data and the image data sequence included in the image file sequence (for example, “STOP001-0.RGB”, “STOP001-1.RGB”, “STOP001-2.RGB”) The difference is compared with a predetermined threshold, and if the difference is equal to or smaller than the predetermined threshold, it is determined that the recording end condition is satisfied. If the difference exceeds the predetermined threshold, another image file sequence ( For example, the difference between the image data and the image data sequence included in “STOP002-0.RGB”, “STOP002-1.RGB”, “STOP002-2.RGB”) is calculated. It may be compared with the difference with a predetermined threshold value.

  As is apparent from the above, according to the fifth embodiment, the image data string that matches the recording start condition and the recording end condition whose settings have been received by the recording condition setting unit 21 and the screen image input unit 2 have acquired. By analyzing the moving image of the image data sequence, the image data sequence is compared with the image data sequence, and the recording start timing and the recording of the image data are ended with reference to the comparison result. Since the recording end timing is detected, the detection accuracy of the recording start timing and the recording end timing can be improved.

  In the fifth embodiment, the recording condition setting unit 21 and the screen image analysis unit 22 are mounted instead of the recording condition setting unit 3 and the screen image analysis unit 4 shown in FIG. Instead of the recording condition setting unit 11 and the screen image analysis unit 12, a recording condition setting unit 21 and a screen image analysis unit 22 may be mounted.

Embodiment 6 FIG.
FIG. 18 is a block diagram showing a screen recording apparatus according to the sixth embodiment of the present invention. In the figure, the same reference numerals as those in FIG.
The compression method storage unit 31 is a recording medium such as a memory that stores a compression method and compression parameters (resolution, frame rate, compression rate) corresponding to the load on the CPU.
The screen image recording processing unit 32 starts recording the image data acquired by the screen image input unit 2 from the time when the recording start timing is detected by the screen image analysis unit 4, and the recording end timing is set by the screen image analysis unit 4. When detected, the recording of the image data is terminated.
However, the screen image recording processing unit 32 measures the load of the CPU when recording of image data is started, reads the compression method and the compression parameter corresponding to the load measurement result from the compression method storage unit 31, and the compression method. The image data is recorded with the compression parameter.
The compression method storage unit 31, the screen image recording processing unit 32, and the image data storage unit 6 constitute an image data recording unit.

  In FIG. 13, each of the screen image input unit 2, the recording condition setting unit 3, the screen image analysis unit 4, the screen image recording processing unit 32, and the image reproduction unit 7 which are components of the screen recording apparatus is dedicated hardware (for example, , A semiconductor integrated circuit on which a CPU is mounted, a one-chip microcomputer, or the like is assumed. However, when the screen recording device is configured by a computer, the screen image input unit 2, recording conditions A program describing the processing contents of the setting unit 3, the screen image analysis unit 4, the screen image recording processing unit 32, and the image reproduction unit 7 is stored in a computer memory, and the CPU of the computer is stored in the memory. The program may be executed.

Next, the operation will be described.
In the first to fifth embodiments, the screen image recording processing unit 5 records the image data with the preset compression method and compression parameters. However, the screen image recording processing unit 32 loads the CPU. The image data may be recorded with the compression method and compression parameter corresponding to the measurement result of the load.
Specifically, it is as follows.

FIG. 19 is an explanatory diagram showing an example of a compression method and compression parameters (resolution, frame rate, compression rate) corresponding to the CPU load.
The compression method storage unit 31 stores a compression method and compression parameters as shown in FIG. 19 in advance.

The screen image recording processing unit 32 starts recording the image data acquired by the screen image input unit 2 from the time when the recording start timing is detected by the screen image analysis unit 4 as in the first to fifth embodiments. Then, when the recording end timing is detected by the screen image analysis unit 4, the recording of the image data is ended.
However, the screen image recording processing unit 32 measures the load of the CPU when starting the recording of the image data, reads the compression method and the compression parameter corresponding to the measurement result of the load from the compression method storage unit 31, and Set the compression method and compression parameters to be used.

For example, if the CPU load is 90%, the compression method is set to JPEG, the resolution is set to 50%, the frame rate is set to 1 fps, and the compression rate is set to 1/20. If the CPU load is 30%, the compression method is set to Set JPEG, resolution 100%, frame rate 5fps, compression rate 1/20.
When the compression method and compression parameters to be used are set, the compression method storage unit 31 records image data in the image data storage unit 6 with the compression method and compression parameters.

  As is apparent from the above, according to the sixth embodiment, the load on the CPU is measured when the screen image recording processing unit 32 starts recording image data, and the measurement result of the load from the compression method storage unit 31 is measured. The compression method and the compression parameter corresponding to the above are read and the image data is recorded with the compression method and the compression parameter. Therefore, the efficiency of the image data recording process can be improved.

  1 application, 2 screen image input section (image data acquisition means), 3, 11, 13, 15, 21 recording condition setting section (condition setting means), 4, 12, 14, 16, 22 screen image analysis section (timing detection) Means), 5, 32 screen image recording processing section (image data recording means), 6 image data storage section (image data recording means), 7 image playback section (image playback means), 31 compression method storage section (image data recording means) ).

Claims (8)

  Image data acquisition means for acquiring image data indicating an image on the screen that changes with the execution of the application, and the recording as a recording start condition when starting recording of the image data acquired by the image data acquisition means Condition setting means for accepting setting of image data that matches a start condition, and accepting setting of image data that matches the recording end condition as a recording end condition when ending recording of the image data, and the condition setting means The image data received by the setting is compared with the image data acquired by the image data acquisition means, the recording start timing for starting the recording of the image data with reference to the comparison result, and the recording of the image data is ended. To detect the recording end timing And when the recording start timing is detected by the timing detection means and the recording end timing is detected by the timing detection means. A screen recording apparatus comprising: image data recording means for ending the recording of the image data; and image reproducing means for reproducing an image on the screen according to the image data recorded by the image data recording means.   The timing detection means calculates the difference between the image data that matches the recording start condition received by the condition setting means and the image data that matches the recording end condition and the image data acquired by the image data acquisition means, and the recording The timing at which the difference between the image data that matches the start condition and the image data falls below a predetermined threshold is detected as the recording start timing, and the difference between the image data that matches the recording end condition and the image data falls below the predetermined threshold. The screen recording apparatus according to claim 1, wherein the timing is detected as a recording end timing.   Image data acquisition means for acquiring image data indicating an image on the screen that changes with the execution of the application, and the recording as a recording start condition when starting recording of the image data acquired by the image data acquisition means Condition setting means for accepting the setting of image data that matches the start condition, image data received by the condition setting means and the image data obtained by the image data obtaining means are compared, and the comparison result is referred to. A timing detection means for detecting a recording start timing for starting the recording of the image data and a recording end timing for ending the recording of the image data, and the image data from the time when the recording start timing is detected by the timing detection means. Acquired by acquisition means Image data recording means for starting the recording of the image data, and ending the recording of the image data when the recording end timing is detected by the timing detecting means, and the image data recorded by the image data recording means. Therefore, a screen recording apparatus comprising image reproducing means for reproducing an image on a screen.   The timing detection unit calculates a difference between the image data set by the condition setting unit and the image data acquired by the image data acquisition unit, detects a timing at which the difference exceeds a predetermined threshold as a recording start timing, 4. The screen recording apparatus according to claim 3, wherein a timing at which the difference becomes equal to or less than a predetermined threshold is detected as a recording end timing. The condition setting means has a function of accepting the setting of the color of the area to be excluded from the comparison target in the image on the screen,
The timing detection unit excludes, from the comparison target, image data of the same color as the color accepted by the condition setting unit from among the image data acquired by the image data acquisition unit. 5. The screen recording apparatus according to claim 1, wherein the remaining image data is compared with the image data whose setting is accepted by the condition setting means. The condition setting means has a function of accepting the setting of the comparison target area in the image on the screen,
The timing detection unit extracts image data of an area whose setting is received by the condition setting unit from the image data acquired by the image data acquisition unit, and the setting is received by the image data and the condition setting unit. 5. The screen recording apparatus according to claim 1, wherein the image data are compared with each other.   The timing detection unit, when the setting of a series of image data is received by the condition setting unit, compares the series of image data with the series of image data acquired by the image data acquisition unit. The screen recording apparatus according to claim 1.   The image data recording means measures the load of the CPU when recording of the image data acquired by the image data acquisition means is started, and records the image data with a compression method and compression parameters corresponding to the measurement result of the load. The screen recording device according to claim 1, wherein the screen recording device is a screen recording device.

Images