JP4519894B2 - Recording device and program for the recording device - Google Patents

Description

  The present invention relates to a recording apparatus that records a captured image obtained by capturing an operation screen, and a program for such a recording apparatus.

  In recent years, with the widespread use of networks such as intranets and local area networks, the number of cases in which the operation status of remote computer terminals is checked for the purpose of monitoring computer terminals connected to the network or for classes using the network has increased. Yes. As one technique for achieving such an object, there is a technique for grasping the operation status of a computer terminal by capturing an operation screen of the computer terminal and confirming the captured image (for example, Patent Document 1). In the remote control system according to Patent Document 1, a remote control server and a client terminal are connected via a communication network, and the remote control server is configured to receive access request information from the client terminal. The client terminal transmits the image data displayed on the monitor to the remote control server, and the remote control server displays the received image data on the monitor, and the captured image recording means of the remote control server captures the screen of the client terminal. The image is recorded in the operation log at a predetermined interval, and after this recording, the log reproduction processing unit of the remote control server reproduces the captured image recorded in the operation log. However, in the technique according to Patent Document 1, the captured image is simply reproduced. For example, when the mouse cursor is in the reproduction screen and the mouse cursor moves quickly, the person who is viewing the reproduction screen Had the problem of losing sight of the mouse cursor.

JP 2006-108947 A (paragraph number 0051, etc.)

  In view of the above problems, an object of the present invention is a recording device in which a captured image obtained by capturing an operation screen is input and recording is performed so that the user's eyes can easily follow the movement of the cursor of the pointing device during playback. It is to provide a device and a program for such a recording device.

  The features of the recording apparatus according to the present invention for achieving the above object are a storage unit that stores a captured image obtained by capturing an operation screen, a display area dividing unit that divides the operation screen into sections, A cursor movement distance calculation unit for calculating a movement distance of the cursor based on cursor position information indicating a cursor position of the pointing device on the operation screen; and a movement distance integration for counting the total movement distance based on the movement distance A target movement distance setting unit that sets a target movement distance of the cursor for each section, and a signal output timing of an extraction signal that is output every time the integrated movement distance reaches the set target movement distance A capture image extraction unit for extracting a capture image from the storage unit; and a capture image case for recording the extracted capture image. In that it comprises a part, a.

  With this configuration, the operation screen is divided into sections by the display area dividing unit, and saved every time the movement distance of the pointing device cursor reaches the target movement distance set for each of the divided sections. Since the captured image stored in the section is extracted, it is possible to easily record the captured image corresponding to the position of the cursor. Therefore, even if the movement of the cursor in the extracted image is irregular, it is possible to reproduce the movement of the cursor according to the target movement distance set for each section by simply reading from the captured image storage unit. This makes it easier to follow the eye without losing sight of the cursor. In addition, since the target movement distance is set for each section, for example, by setting a large target movement distance in a section with low importance, it is possible to reduce the number of captured images to be extracted. Therefore, the recording capacity can be reduced. The storage unit according to the present invention includes a storage unit having a function of temporarily storing a captured image and a storage unit that fixes the captured image as data, such as a DVD or a hard disk.

  In the recording apparatus, it is preferable that the target moving distance setting unit changes the target moving distance according to a display object displayed in a section divided for each section. With such a configuration, when there are icons to be watched as operation objects, application windows, or the like as display objects in a predetermined section, more captured images are extracted by setting a small target moving distance. be able to. Therefore, the movement of the cursor can be reproduced in detail during reproduction, and the movement of the cursor can be viewed more carefully.

  The recording device according to the present invention is characterized in that a storage unit that stores a captured image obtained by capturing an operation screen, a display area dividing unit that divides the operation screen into sections, and the operation screen A cursor movement distance calculation unit that calculates a movement distance of the cursor based on cursor position information indicating a cursor position of the pointing device, a movement distance coefficient determination unit that determines a movement distance coefficient for each section, and the movement distance And a moving point calculation unit that calculates a moving point based on the moving distance coefficient, and the storage based on a signal output timing of an extraction signal that is output each time the moving point reaches a preset extraction point. Capture image extraction unit for extracting a capture image from the unit, and capture image storage for recording the extracted capture image It lies in having the, and.

  With such a configuration, it is possible to easily record a captured image according to the movement point calculated based on the movement distance and the movement distance coefficient of the cursor. Therefore, even if the movement of the cursor in the acquired image is irregular, it is possible to reproduce the movement of the cursor according to the movement distance coefficient set for each section by simply reading from the captured image storage unit. This makes it easier to follow the eye without losing sight of the cursor. Further, since the movement distance coefficient is set for each section, for example, by setting a large movement distance coefficient in a section having low importance, it is possible to reduce the number of captured images to be extracted. Therefore, the recording capacity can be reduced.

  Furthermore, in the present invention, a program for a recording apparatus is also included in the scope of rights, and the computer program stores a captured image obtained by capturing an operation screen in a storage unit, and the operation screen for each section. A display area dividing function for dividing, a cursor moving distance calculating function for calculating a moving distance of the cursor based on cursor position information indicating a cursor position of the pointing device on the operation screen, and an integration based on the moving distance A moving distance integrating function that counts moving distances, a target moving distance setting function that sets a target moving distance of the cursor for each section, and an extraction that is output every time the integrated moving distance reaches a set target moving distance Capture image extractor for extracting a capture image from the storage unit based on a signal output timing of a signal When a captured image storage function of recording the captured image in which the extracted to the captured image storage unit, it is possible to realize the computer.

  In the present invention, another program for the recording apparatus is also included in the scope of rights, and the computer program defines a storage function for storing a captured image obtained by capturing an operation screen in a storage unit, and the operation screen. A display area dividing function that divides every time, a cursor moving distance calculating function that calculates a moving distance of the cursor based on cursor position information indicating a cursor position of the pointing device on the operation screen, and a movement for each section A moving distance coefficient determining function that determines a distance coefficient, a moving point calculating function that calculates a moving point based on the moving distance and the moving distance coefficient, and each time the moving point reaches a preset extraction point Based on the signal output timing of the extracted signal to be output, the captured image is extracted from the storage unit And Yapucha image extracting function, and capture an image storage function of recording the captured image in which the extracted to the captured image storage unit, it is possible to realize the computer.

  Compared with the recording apparatus as the subject of the present invention described above, the programs for these recording apparatuses are not substantially different in characteristic configuration, and the above-described operational effects can be obtained.

[First embodiment of the present invention]
Hereinafter, as an example in the case of using the recording apparatus according to the present invention, a description will be given using a computer network system configured as shown in FIG. FIG. 1 is a block diagram schematically showing this computer network system. A terminal 1 on which various files are created, viewed, copied, modified, deleted, printed, etc. by a user, and the operation of this terminal 1 are shown. And a recording device 2 for recording the situation. When the user uses the terminal 1, the captured image based on the operation screen displayed on the monitor 1A of the terminal 1, the display object information of the display object displayed on the captured image, and the operation of the user's pointing device Based on the cursor position information recorded based on the user's operation, the operation log information recording the operation status based on the user's operation is transmitted from each terminal 1 to the recording device 2. Therefore, it may be considered that the operation screen displayed on the monitor 1 </ b> A of each terminal 1 is recorded in the recording device 2. In the present embodiment, the captured image transmitted from each terminal 1 is first temporarily stored in the temporary storage unit 27 (see FIG. 2). Of the captured images temporarily stored in the temporary storage unit 27, the captured images satisfying predetermined conditions are stored in the captured image storage unit 10 based on various processes performed by the functional units of the recording apparatus 2. It becomes the composition which is done. Although details will be described later, in the present embodiment, the temporary storage unit 27 corresponds to the storage unit in the claims of the present application. Note that the capture image is generated by a capture image generation unit (not shown) provided in the terminal 1. In this embodiment, the captured image, the display object information, the cursor position information, and the operation log information are transmitted separately, but the present invention is not limited to this. Naturally, it is also possible to employ a configuration in which operation information including these captured images, display object information, cursor position information, and operation log information is transmitted to the recording apparatus 2 at once.

  FIG. 2 is a block diagram schematically showing the configuration of the recording apparatus 2. The recording apparatus 2 records a captured image extracted from a captured image temporarily stored in the temporary storage unit 27 and a temporary storage unit 27 that temporarily stores the captured image acquired by the terminal 1 connected to the network. A capture image storage unit 10 and a cursor position information storage unit 11 that stores cursor position information of the pointing device are provided.

  The cursor position information includes position information of the cursor Cu recorded based on the user's cursor operation on the terminal 1. The recording apparatus 2 also includes a display object information storage unit 12 that stores display object information. This display object information includes icon information indicating the coordinates of icons displayed on the monitor 1A of the terminal 1, task bar information indicating the coordinates of the task bar including the start button, and the like. In addition, an icon, a task bar, etc. are defined as the display thing displayed on a captured image. Further, the display object information includes information on the position, content, importance, etc. of the display object, and can be used as a position of the display object to specify a predetermined position described later. Furthermore, the recording apparatus 2 includes an operation log information storage unit 13 that stores operation log information. The operation log information includes an operation log recorded based on the operation content, operation time, etc. of the user who uses the terminal 1. Therefore, the user's operation status can be recognized from the operation log information.

  In the cursor position information, the position information of the cursor Cu at the terminal 1 is recorded, and the acquisition time when the position information is acquired and the x and y coordinates of the cursor Cu at the acquisition time are recorded. In addition, the acquisition time when the respective coordinates are acquired and the x-coordinate and the y-coordinate at the acquisition time are also recorded in the icon information and task bar information. These coordinates are the coordinates when displayed on the monitor 1A.

  The cursor movement distance calculation unit 14 calculates the movement distance by which the cursor Cu has moved within a predetermined time from the cursor position information stored in the cursor position information storage unit 11. The calculated travel distance is recorded in the travel distance calculation result recording unit 15 as a travel distance calculation result. Further, based on the movement distance calculation result recorded in the movement distance calculation result recording unit 15, the movement distance integration unit 16 counts the integrated movement distance of the cursor Cu.

  The recording device 2 has a function of recording a captured image so that the user's eyes viewing the playback screen can easily follow the movement of the cursor Cu displayed on the playback screen when the recorded captured image is played back on the monitor 2A. I have. This can be realized by recording the operation screen at that time each time the cursor Cu moves a predetermined distance (a constant distance). This predetermined distance can be set in advance by the target movement distance setting unit 17 as the target movement distance. The target moving distance is set by the target moving distance setting unit 17 for each section in which the operation screen on the terminal 1A is divided by the display area dividing unit 18. The display area dividing unit 18 will be described later.

  The movement distance comparison unit 19 compares the accumulated movement distance counted by the movement distance accumulation unit 16 with the target movement distance set by the target movement distance setting unit 17 and outputs the comparison result to the extraction signal output unit 20. Based on the comparison result, the extraction signal output unit 20 outputs to the capture image extraction unit 21 an extraction signal that is the timing at which the capture image extraction unit 21 extracts the capture image from the temporary storage unit 27. This extraction signal is output every time the cumulative movement distance of the cursor Cu reaches the target movement distance set by the target movement distance setting unit 17. Each time the capture image extraction unit 21 receives the extraction signal, the capture image extraction unit 21 extracts a capture image corresponding to the extraction signal from the temporary storage unit 21 and records it in the capture image storage unit 10.

  The position evaluation value calculation unit 22 included in the recording device 2 calculates the position evaluation value of the cursor Cu based on the cursor position information and display object information such as icon information and task bar information. This position evaluation value is a value that defines the relationship between the position of the cursor Cu and the position of the display object, and specifically increases if the cursor Cu is on a display object such as an icon or task bar. For example, when the cursor Cu is on the icon, there is a high possibility that it is a previous stage such as an application activation or a folder opening operation, and when it is on the task bar, there is a possibility that it is a previous stage such as an application activation. high. As described above, when the cursor Cu is on a predetermined position (designated display object) set in advance, that is, when the cursor position coincides with the predetermined position, the importance can be increased because there is a possibility of an important operation. It is. Therefore, such an operation can be predicted in advance based on the position evaluation value. On the other hand, when the cursor Cu is not positioned on the icon or the task bar, either the cursor Cu is stationary on the other area or moved on the other area. Therefore, it can be predicted that no application or file operation has been performed.

  The extraction signal output unit 20 additionally outputs an extraction signal to the capture image extraction unit 21 based on the position evaluation value calculated by the position evaluation value calculation unit 22. That is, even if the integrated movement distance does not reach the target movement distance, if it is determined that the cursor Cu is positioned on the icon or the task bar, the extraction signal output unit 20 extracts the extracted signal as a captured image. To the unit 21. The operation log information stored in the operation log information storage unit 13 is input to the operation log determination unit 24. Even when the operation log determination unit 24 determines that a specific operation such as activation of an application has been performed on the terminal 1, the extraction signal output unit 20 additionally outputs an extraction signal to the capture image extraction unit 21. .

  As described above, the captured image transmitted from the terminal 1 is first temporarily stored in the temporary storage unit 27. In the captured image temporarily stored in the temporary storage unit 27, only the captured image corresponding to the extraction signal output from the extraction signal output unit 20 is extracted by the capture image extraction unit 21 and recorded in the capture image storage unit 10. Is done. The captured image that is not recorded in the captured image storage unit 10 and remains in the temporary storage unit 27 is preferably deleted after a predetermined time has elapsed. Of course, the predetermined time may be immediately after the extraction process elapses, or may be after several hours or days. As described above, the capture image extraction unit 21 extracts the capture image temporarily stored in the temporary storage unit 27 based on the extraction signal output from the extraction signal output unit 20.

  Here, when the extraction signal is output from the extraction signal output unit 20, the integrated movement distance corresponding to the cursor position information that has output the extraction signal is larger than the target movement distance set by the target movement distance setting unit 17. In this case, since the captured image corresponding to the signal output timing of the extracted signal is not in the temporary storage unit 27, it cannot be extracted. For this reason, the recording apparatus 2 is provided with an interpolation function for interpolating and generating captured images that are insufficient in such a case. When the captured image corresponding to the extracted signal output by the extracted signal output unit 20 is not in the temporary storage unit 27, the temporary storage unit 27 sends the captured image corresponding to the extracted signal to the captured image extraction unit 21. A signal indicating that there is no data is transmitted. In response to this signal, the capture image extraction unit 21 instructs the interpolation screen generation unit 23 to generate an interpolation screen. In accordance with this command, the interpolation screen generator 23 generates an interpolation screen. The interpolation screen generation unit 23 generates an interpolation screen as follows. First, the interpolation screen generation unit 23 determines the coordinates of the cursor Cu corresponding to the signal output timing of the extraction signal from the cursor position information recorded in the cursor position information storage unit 11. Next, the background image is generated by deleting the cursor Cu from any of the images before and after the corresponding time by image processing. Then, an image of the cursor Cu is synthesized with this background image so as to correspond to the determined coordinates. In this way, an interpolation screen is generated. The generated interpolation screen is recorded in the captured image storage unit 10 so as to be appropriately inserted into the captured image extracted from the temporary storage unit 27. Therefore, even when the captured image corresponding to the extracted signal is not in the temporary storage unit 27, the recording device 2 can record an image in which the cursor Cu moves smoothly by generating an interpolation screen. Become.

  The captured image and the interpolation screen recorded in the captured image storage unit 10 are transmitted to the image reproduction unit 25 and reproduced as a reproduction screen on the monitor 2A via a VRAM (Video Random Access Memory) 26. Further, the recording apparatus 2 has a function unit for performing various operations related to recording processing and reproduction thereof with hardware or software, or both, with the CPU as a core member.

  The display area dividing unit 18 divides the operation screen into sections. An example in which the display area is divided into sections is shown in FIG. In FIG. 3, the x coordinate and y coordinate of the operation screen are each divided into four parts and divided into 16 sections. However, the present invention is not limited to this and is equally divided. It is not limited to. The target moving distance setting unit 17 sets the target moving distance for each of the divided sections. An example is shown in FIG. As shown in FIG. 4, the target moving distance is set for all the sections such that the target moving distance of the section 1 is “200” dots and the target moving distance of the section 2 is “400” dots. Here, since the coordinate information is acquired in units of dots, the unit of the movement distance and the integrated movement distance is also “dot”. Note that this unit is not limited to “dot”, and the length (for example, inches or cm) on the screen can also be used.

  Next, a description will be given using a captured image obtained by capturing an operation screen. FIG. 5 is an example of an image obtained by capturing an operation screen of the monitor 1 </ b> A based on the operation of the terminal 1 every predetermined time. The cursor Cu moves from the screen (a) acquired during the acquisition time t1 to t4 to the screen (d), but from the screen (d) during the acquisition time t4 to t5. As shown on the screen, the movement of the cursor Cu is not seen. Similarly, during the acquisition time t5 to t11, the cursor Cu is moved from the screen (e) to the screen (k), but from the time t11 to t12, the screen (k) No movement of the cursor Cu is seen as shown on the screen from (l) to (l). Such a captured image is transmitted as operation screen information to the recording apparatus 2 and temporarily stored in the temporary storage unit 27.

  Further, the cursor position information is stored in the cursor position information storage unit 11. An example of this cursor position information is shown in FIG. The cursor position information is information indicating the position of the cursor Cu on the operation screen, and includes acquisition time when the position information of the cursor Cu is acquired, and coordinate information including x and y coordinates at the acquisition time. In the example of FIG. 6, the acquisition time is t1 to t12. The acquisition interval is set every 2 seconds in accordance with the capture timing, but is not limited thereto. Depending on the setting, it is possible to make it 2 seconds or less, and naturally it is possible to make it 2 seconds or more. Regarding the x coordinate and the y coordinate, the lower left corner of the operation screen is set as the reference coordinate with the x coordinate being 0 and the y coordinate being 0, but is not limited thereto. Of course, it is possible to record the coordinates of the cursor Cu using the predetermined location as the reference coordinates. In this example, the operation screen area is described as an example of a display area of 1024 × 768 (dot).

  Based on the cursor position information stored in the cursor position information storage unit 11, the cursor movement distance calculation unit 14 calculates the movement distances of the x and y coordinates of the cursor Cu. These calculation results are recorded in the movement distance calculation result recording unit 15. FIG. 7 shows the movement distance calculation result calculated from FIG. The movement distance “x1−x2” of the x coordinate between the acquisition times t1 and t2 is calculated and recorded as “−165”. In the same manner, “x2-x3” and thereafter are sequentially calculated and recorded. Furthermore, the movement distance of the y coordinate is recorded in the same manner as the calculation of the x coordinate. Further, the result of calculating the movement distance on the operation screen from the movement distance of the x coordinate and the movement distance of the y coordinate is recorded as the movement distance. However, here, the calculation result of the movement distance shows the result of rounding off the decimal point for easy viewing.

  The travel distance integrating unit 16 counts the integrated travel distance of the cursor Cu based on the travel distance calculation result recorded in the travel distance calculation result recording unit 15. The movement distance comparison unit 19 compares the integrated movement distance and the target movement distance set in the target movement distance setting unit 17, and the extraction signal output unit 20 extracts the extraction signal every time the integration movement distance reaches the target movement distance. (High signal) is output. FIG. 8 shows the timing at which the extraction signal is output based on the cumulative movement distance counting result counted from the movement distance calculation result of FIG.

  In FIG. 8, the integrated movement distance from the acquisition time t1 to t2 is “170” dots. Since the section where the cursor Cu is located during this acquisition time is the movement within “section 11” in FIG. 3, the section where the cursor is located is shown as “section 11 → section 11” in FIG. . Further, since the target moving distance in the section is within the same section, “300” dot → “300” dot ”. The accumulated movement distance of the cursor Cu at “t1-t2” is “170” dots, but since the target movement distance (“300” dots) has not been reached, no extraction signal is output. Therefore, the extraction signal is “Low”. Then, when the travel distance “130” dot from the acquisition time t2 to t3 is added to this cumulative travel distance, the cumulative travel distance from the acquisition time t1 to t3 is obtained as “300” dot. In this case (that is, when t3 is reached), since the integrated movement distance has reached the target movement distance, a High signal is output as the extraction signal. Note that in the state of the acquisition time t3, the target movement distance and the integrated movement distance are equal, and therefore, the captured image acquired at the acquisition time t3 exists in the temporary storage unit 27. Therefore, the capture image corresponding to the extraction signal is extracted by the capture image extraction unit 21 and recorded in the capture image storage unit 10. Further, the accumulated moving distance of the moving distance integrating unit 16 is reset in accordance with the output of this extraction signal. Therefore, the cumulative movement distance count after the acquisition time t3 starts from “0”.

  The accumulated movement distance from the acquisition time t3 to t4 is “300” dots. During this acquisition time, the cursor Cu moves from “section 10” to “section 5” via “section 6”. Therefore, the target moving distance also changes from “300” dot → “300” dot → “200” dot ”. In this case (that is, when t4 is reached), the integrated movement distance greatly exceeds the target movement distance “200” dots. Therefore, the captured image captured at this timing (acquisition time t4) does not exist in the temporary storage unit 27. Therefore, a command to generate an interpolation screen is issued from the capture image extraction unit 21 to the interpolation screen generation unit 23. In response to this command, the interpolation screen generator 23 generates an interpolation screen as described above. When the interpolation screen is generated, a value (“100” dots) obtained by subtracting the target movement distance (here “200” dots) from the total movement distance (here “300” dots) is accumulated. It remains as a moving distance.

  Since the moving distance from the acquisition time t4 to t5 is “0” dots, the integrated moving distance “100” dots will continue to remain. In this case, since the integrated movement distance has not reached the target movement distance, the extraction signal becomes Low.

  In the case of the acquisition time t6, since the integrated movement distance is “245” dots and greatly exceeds the target movement distance “200” dots, there is no captured image, so an interpolation screen is generated. Then, at the acquisition time t6, the extraction signal is output again. This is because the extraction signal is output because the cursor Cu is on the icon (see FIG. 5F). The position of this icon can be determined from the display object information recorded in the display object information storage unit 12, and the position evaluation value is based on the display object information and the cursor position information recorded in the cursor position information storage unit 11. When the calculation unit 22 determines, the extraction signal output unit 20 outputs the extraction signal.

  Thereafter, the processing is continued in the same manner, and the cumulative movement distance is counted until the extraction signal is output. The accumulated movement distance is “205” dots between the acquisition times t7 and t8, but in this case, the difference within a predetermined range (in this example, “± 10” dots is set to a predetermined value, although exceeding the target movement distance). Therefore, the extraction signal is output when the acquisition time t8 is reached. The setting of the predetermined range can be set in advance or can be changed. Further, since the cursor Cu is positioned on the icon also at the acquisition time t9, an extraction signal is output.

  Next, the accumulated movement distance from the acquisition time t10 to t11 is counted. The accumulated movement distance remaining at t10 (“150” dot) and the movement distance of the cursor Cu at “t10-t11” (“300”). dot) is treated as the accumulated travel distance (“450” dots). Also in this case, since the integrated movement distance greatly exceeds the predetermined range (“± 10” dots) with respect to the target movement distance, there is no captured image in the temporary storage unit 27. Therefore, an interpolation screen generation command is output from the capture image extraction unit 21 to the interpolation screen generation unit 23, and the interpolation screen generation unit 23 generates an interpolation screen.

  For example, in the case where the acquisition time ends in t12 in FIG. 8, even if the integrated movement distance does not reach the target movement time, the extraction screen is output and recorded as a final screen. It is also possible.

  Next, an image recorded in the captured image storage unit 10 based on the extracted signal will be described. FIG. 9 is an image recorded in the captured image storage unit 10. FIG. 9A is an image at the acquisition time t1. Next, since the extraction signal is output at the acquisition time t3, an image at the acquisition time t3 is recorded in the capture image storage unit 10 (see FIG. 9B). 9C and 9D, the interpolation screen generated by the interpolation screen generation unit 23 is recorded. In FIG. 9E, since the cursor Cu is on the icon, it is recorded although the target movement distance has not been reached according to the evaluation of the position evaluation value calculation unit 22. Similarly, the captured image acquired at the acquisition time corresponding to the output of the extraction signal is recorded in the captured image storage unit 10.

  In FIG. 9 (i), the target moving distance is not reached at the acquisition time t12, but is recorded as a captured final screen. Thus, since the operation screen can be recorded according to the target movement distance set for each section, the movement distance of the cursor Cu can be reproduced so that the cursor Cu moves at a constant speed in each divided area. Can do. Therefore, it is possible to easily view the playback screen without losing sight of the cursor Cu. When the moving distance of the cursor Cu is reproduced according to the target moving distance by the recording apparatus 2, the operation screen may be fast-forward reproduction or slow reproduction from the viewpoint of reproduction speed. In such a case, although not shown, it is possible to recognize whether or not the playback speed has been changed for the person watching the monitor 2A during fast forward playback or slow playback. As described above, the distinction can also be clearly indicated by a counter display or a lamp display.

  Next, the present embodiment will be described using a flowchart. FIG. 10 is a flowchart for recording the operation screen according to the target movement distance set by the recording apparatus 2 for the movement of the cursor Cu. First, the operation screen on the terminal 1A is divided into sections (step # 01). Then, the target movement distance setting unit 17 sets a target movement distance for each divided section (step # 02). At this time, it is also possible to set a target movement distance desired by the user by a target movement distance input unit (not shown). Next, the position evaluation value calculation unit 22 extracts the cursor position information stored in the cursor position information storage unit 11 (step # 03).

  Then, the position evaluation value calculation unit 22 determines whether or not the cursor Cu is on the predetermined position based on the position evaluation value. The result is output to the extracted signal output unit 20. The position information of the display object is stored in the display object information storage unit 12 as display object information. If the cursor Cu is on a predetermined position such as a display object (step # 04: No), the extraction signal output unit 20 outputs the extraction signal to the capture image extraction unit 21 (step # 12). On the other hand, if the cursor Cu is not on the predetermined position (step # 04: Yes), the operation log determination unit 24 extracts the operation log information stored in the operation log information storage unit 13 (step # 05), and the program is executed. It is confirmed whether or not it has been activated, and the determination result is output to the extraction signal output unit 20. If the program is activated (step # 06: No), the extraction signal output unit 20 outputs the extraction signal to the capture image extraction unit 21 (step # 12). On the other hand, if the program is not activated (step # 06: Yes), the process proceeds to the next process.

  In the above processing, if the cursor Cu is not on the predetermined position and the program is not activated (step # 06: Yes), the cursor movement distance calculation unit 14 stores the cursor stored in the cursor position information storage unit 11. Based on the position information, the movement distance of the cursor Cu is calculated (step # 07). The calculation result of the movement distance is recorded in the movement distance calculation result recording unit 15 (step # 08).

  Next, the travel distance integrating unit 16 counts the integrated travel distance based on the travel distance calculation result (step # 09), and the obtained integrated travel distance and the target travel distance set by the target travel distance setting unit 17 The movement distance comparison unit 19 performs comparison using (Step # 10). If the target movement distance and the accumulated movement distance are within the predetermined range (step # 11: Yes), the extraction signal output unit 20 outputs the extraction signal to the movement distance accumulation unit 16. In response to the extraction signal, the travel distance integrating unit 16 resets the integrated travel distance (step # 11A). The extraction signal is also output to the captured image extraction unit 21 (step # 12).

  The capture image corresponding to the cursor position information for which the extraction signal is output is extracted from the temporary storage unit 27 by the capture image extraction unit 21 and recorded in the capture image storage unit 10 (step # 13). Returning to step # 11, if the integrated movement distance is smaller than the target movement distance (step # 15: No), it is confirmed whether or not the movement of the cursor Cu is a movement to another section. If the movement of the cursor Cu is not a movement to another section (step # 18: Yes), the next cursor position information is extracted (step # 03), and the same processing as described above is continued. On the other hand, if the movement of the cursor Cu is a movement to another section (step # 18: No), the target movement distance of the section to which the cursor Cu is moved is read and set as the target movement distance (step #). 19). And it returns to step # 03 and a process is continued.

  If the integrated travel distance is greater than the target travel distance in step # 15 (step # 15: Yes), the operation screen corresponding to the target travel distance is not temporarily stored in the temporary storage unit 27. Since the operation screen needs to be generated by interpolation, the capture image extraction unit 21 issues an interpolation screen generation command to the interpolation screen generation unit 23 (step # 16). In response to this generation command, the interpolation screen generation unit 23 generates an interpolation screen from the cursor position information and the captured images before and after the cursor position information, and the movement distance integration unit 16 recalculates the integration movement distance (step # 17). . Then, the interpolation screen is recorded in the captured image storage unit 10 (step # 13).

  Returning to step # 13, if there is no unevaluated cursor position information in the cursor position information storage unit 11 (step # 14: Yes), the process is terminated. In this case, the captured image corresponding to the cursor position information determined last may be recorded in the captured image storage unit 10 regardless of whether or not the integrated movement distance has reached the target movement distance. Good.

  If there is unprocessed cursor position information at step # 14 (step 14: No), the process at step # 18 is continued. According to the above flow, the operation screen can be recorded based on the target movement distance of the cursor Cu set for each section, so that the problem of losing sight of the cursor Cu when reproducing the recorded operation screen does not occur. Can be.

[Second embodiment of the present invention]
In the above-described embodiment, the display object information and the cursor position information are described as being transmitted from the terminal 1. As another embodiment according to the present invention, at least one of display object information and cursor position information is configured to calculate position coordinates by performing image processing (image evaluation) on a captured image transmitted from the terminal 1. It is also suitable to do. With such a configuration, the display object information or the cursor position information can be calculated from the captured image transmitted from the terminal 1, so that it is not necessary to acquire such information. In such a case, it is preferable that the above-described operation screen information is not configured only from the captured image but includes various information such as terminal information.

[Other Embodiments]
In the above embodiment, the display object information has been described as being composed of icon information representing the coordinates of the icon and task bar information representing the coordinates of the task bar. However, the present invention is not limited to this. In addition to these, it is naturally possible for the user to specify an arbitrary location on the screen. Even in such a configuration, when the cursor Cu is at the designated position, the extraction signal output unit 20 outputs the extraction signal according to the position evaluation value calculated by the position evaluation value calculation unit 22. Is possible.

  In the above embodiment, the target moving distance has been described as being set in the target moving distance setting unit 17 or can be changed by inputting the target moving distance desired by the user with the target moving distance input unit. Not limited to this. For example, it is preferable that the change can be made according to the time when the captured image was captured. With such a configuration, for example, when the capture time to be recorded is one day, the target moving distance is automatically increased during the break time, or when it is one week, Since it is possible to automatically increase the target movement distance, it is possible to appropriately set according to the recording purpose.

  In the above-described embodiment, the capture image extraction timing is changed based on the target movement distance of the cursor Cu. However, the present invention is not limited to this. For example, it is also possible to input a desired recording time, obtain the movement distance of the cursor Cu from the recording time, and record based on the movement distance. In such a case, the target movement distance of the cursor Cu is obtained based on the input recording time and the movement distance of the cursor, and recording is performed within the desired recording time by recording according to the target movement distance. It becomes possible.

  In the above embodiment, the operation log determination unit 24 checks whether or not there is an operation log indicating the start of the program in the operation log information extracted from the operation log information storage unit 13, and if there is such an operation log, it is extracted. Although it has been described that a signal is additionally output, the present invention is not limited to this. In addition to the activation of the program, it is naturally possible to additionally output an extraction signal even when there is an operation log indicating a specific operation such as a file operation such as file deletion, movement or copying.

  In the above-described embodiment, the target movement distance setting unit 17 has been described as setting the target movement distance short when there is an icon as a display object in the section divided by the display area dividing unit 18, but this is not limitative. Absent. For example, it is naturally possible to change the target movement distance according to the number of icons located in the section. Further, the division for each section is not limited to a quadrangle as shown in FIG. 3, but can be performed by a polygon, a circle, or the like.

  In the above-described embodiment, the extraction signal is described as being output every time the integrated movement distance reaches the target movement distance set for each section, but the present invention is not limited to this. For example, it is naturally possible to configure so that an extraction signal is output when the cursor Cu crosses the section.

  In the above-described embodiment, it has been described that it is determined whether or not the cursor Cu is moved to another section based on the cursor position information in step # 18 of the flowchart, but this is not restrictive. For example, it is possible to identify the section from the cursor position information and acquire the target movement distance of the section.

  In the above embodiment, the capture image is extracted based on the movement distance of the cursor Cu. However, the present invention is not limited to this. For example, the moving distance of the cursor Cu may be replaced with a point value (moving point: P), and a captured image may be extracted according to the point value. In such a case, the movement distance coefficient determination unit sets the movement distance coefficient K for each section, and the movement point calculation unit calculates the movement distance coefficient K and the movement distance of the cursor Cu calculated by the cursor movement distance calculation unit 14. Accordingly, for example, a movement point P as indicated by P = K × movement distance is calculated. The value of the moving distance coefficient K may be changed for each section so that the captured image is extracted from the temporary storage unit 27 every time a predetermined extraction point set in advance is reached. Here, the extraction point is a point used for determining the timing for extracting the captured image, and corresponds to the target movement distance according to the first embodiment described above. Further, the extraction point can be set in advance so as to be constant in the operation screen, and the timing for extracting the captured image can be changed by setting the movement distance coefficient K as a parameter for each section. For example, when the movement distance coefficient K = 0.5 is set for a section with low importance, the movement distance coefficient K = 1 is set for a section with high importance, and the extraction point is set to 100, the movement point P = When the captured image is extracted at 100, the captured image is extracted every movement distance = 100 in the high importance section, whereas the captured image is extracted every movement distance = 200 in the low section. With this configuration, it is possible to reduce the number of captured image extractions in a less important section than in a higher section at the same movement distance. Therefore, the recording capacity can be reduced.

  In the recording apparatus 2 according to the present invention, it is preferable that the display area dividing unit 18 divides based on the position of the display object displayed on the operation screen. With such a configuration, for example, when a display information area in which an icon or the like as a display object is displayed becomes large, the target movement distance is set in order to divide into sections excluding the display information area. Can do. Further, even when an application window is open, it can be divided into sections except for the application area.

  In the recording apparatus 2 according to the present invention, it is preferable that the captured image and the cursor position information are input separately and linked with time based on a common time stamp. With such a configuration, since the captured image and the cursor position information are linked with a common time stamp, even if they are input separately, it is possible to perform processing with matching over time.

  In each of the above-described embodiments, the capture image transmitted from each terminal 1 has been described as being temporarily stored in the temporary storage unit 27. However, the present invention is not limited to this. For example, the captured image, the display object information, and the cursor position information may be configured to be transmitted (input) to the recording apparatus 2 using a storage unit such as a DVD or a hard disk. In such a case, a storage unit such as a DVD or a hard disk functions in the same manner as the temporary storage unit 27, and an extracted signal output from the captured image recorded on the DVD or hard disk according to the processing of the recording device 2. It is naturally possible to record in the captured image storage unit 10 in accordance with the output timing. Therefore, for example, the processing according to the present invention can be performed at an appropriate time after storage in a storage unit such as a DVD or a hard disk. In such an embodiment, a DVD, a hard disk or the like corresponds to the storage unit in the claims of the present application.

  In the above embodiment, the display area dividing unit 18 has been described as dividing the display area into a plurality of sections based on the display object. However, the scope of application of the present invention is not limited to this. For example, the display area can be divided into a display information area in which a display object is displayed and an area other than the display information area. As shown in FIG. 11, the area is divided into an area where icons are arranged (referred to as an icon area) and an area excluding the icon area (referred to as a non-icon area). When the cursor Cu is positioned in the icon area, the application or file operation is likely to be performed. Therefore, the target moving distance can be set short, and the non-icon area is divided. A target moving distance can be set for each section. Furthermore, when the icon area becomes larger as shown in FIG. 12 from the state set as shown in FIG. 11, the target moving distance can be reset in the non-icon area. Alternatively, when the area where the display object is displayed changes due to the activation of the application or the like, it is possible to change the section to be divided and the target movement distance accordingly.

  In the above embodiment, the position evaluation value calculation unit 22 has been described as calculating the position evaluation value. However, the position evaluation value calculation unit 22 may further calculate the display area importance in addition to the position evaluation value. Naturally, it is possible to adopt a simple configuration. This display area importance is a value indicating the importance of the display area based on the display object information. Specifically, the display area importance of the display area where the display object is present increases and changes according to the display object information. Then, the target moving distance setting unit 17 can change the setting of the target moving distance of the section according to the importance of the display area. For example, as shown in FIG. 13, when there is an icon or the like as a display object in a section (for example, section 1, section 5, and section 9) divided by the display area dividing unit 18, the display object exists. The display area importance of the display area is increased. Therefore, the target moving distance setting unit 17 sets the target moving distance short according to the importance of the display area.

  In each of the above-described embodiments, the capture image, the display object information, and the cursor position information are described as being transmitted from each terminal 1 to the recording device 2 via the network. For example, a recording function of the captured image by the recording device 2, that is, the moving distance integrating unit 16, the position evaluation value calculating unit 22, the moving distance comparing unit 19, the extracted signal output unit 20, the captured image extracting unit 21, and the interpolation screen generating unit 23 It is naturally possible to make the terminal 1 itself into a terminal with a stand-alone capture image recording function by modularizing such functions into applications that can be executed on the terminals 1 and installing the applications on the terminals 1.

  In each of the above embodiments, the recording apparatus 2 has been described as being constructed by a single computer, but the present invention is not limited to this. The functions of the recording device 2 may be divided and distributed to a plurality of computers. For example, the captured image storage unit 10 in the recording device 2 is distributed to a dedicated data storage device such as a file server, and the dedicated data storage device Of course, it is also possible to read out display object information and cursor position information and perform various processes.

  In the above embodiment, the capture image has been described as a still image file obtained by capturing an operation screen or a collection of still image files, but is not limited thereto. Of course, the captured image transmitted from the terminal 1 may be a still image or a moving image. Even when the captured image is a moving image, it is naturally possible to calculate the position evaluation value from the display object information and the cursor position information and reduce the recording capacity according to the position evaluation value. Note that when the terminal 1 transmits the moving image in the form of a moving image, the format portion of the terminal 1 is provided with a function of converting a still image into a moving image. Also, when the operation screen (capture screen) of the terminal 1 is displayed on the monitor 2A, it may be a continuous display of a still image (style like a slide show) or a moving image display, and an appropriate still image- A moving image program or a moving image-still image program is implemented. Furthermore, various types of capture images can be selected when recorded in the capture image storage unit 15. For example, JPEG or GIF is used for still images, and MPEG or AVI is used for moving images.

Diagram showing a computer network system Block diagram schematically showing the configuration of the recording apparatus The figure which shows an example which divided | segmented the display area for every division The figure which shows an example of the target moving distance set for every division The figure which shows an example of the image which captured the operation screen The figure which shows an example of cursor position information The figure which shows the movement distance calculation result which is calculated based on cursor position information The figure which shows the total movement distance count result and the extraction signal The figure which shows the operation screen recorded based on the extraction signal Flowchart diagram for recording according to the movement distance of the cursor The figure which shows an example divided | segmented for every division except the icon area | region The figure which shows an example which re-divided for every division when the icon area became large The figure which shows an example of the target movement distance reset when there is an icon in a division

Explanation of symbols

2: Recording device 2A: Monitor 10: Captured image storage unit 11: Cursor position information storage unit 12: Display object information storage unit 13: Operation log information storage unit 14: Cursor movement distance calculation unit 15: Movement distance calculation result recording unit 16 : Moving distance integrating unit 17: target moving distance setting unit 18: display area dividing unit 19: moving distance comparing unit 20: extraction signal output unit 21: captured image extracting unit 22: position evaluation value calculating unit 23: interpolation screen generating unit 24 : Operation log determination unit 25: Image reproduction unit 26: VRAM
27: Temporary storage unit (storage unit)

Claims (6)

  A storage unit that stores a captured image obtained by capturing an operation screen, a display area dividing unit that divides the operation screen into sections, and cursor position information that indicates a cursor position of the pointing device on the operation screen. A cursor movement distance calculation unit that calculates a movement distance of the cursor based on the movement distance integration unit that counts an integrated movement distance based on the movement distance, and a target movement that sets a target movement distance of the cursor for each section A distance setting unit; a capture image extracting unit that extracts a captured image from the storage unit based on a signal output timing of an extraction signal that is output every time the integrated movement distance reaches a set target movement distance; and the extraction And a capture image storage unit for recording the captured image.   The recording apparatus according to claim 1, wherein the target moving distance setting unit changes the target moving distance according to a display object displayed in a section divided for each section.   A storage unit that stores a captured image obtained by capturing an operation screen, a display area dividing unit that divides the operation screen into sections, and cursor position information that indicates a cursor position of the pointing device on the operation screen. Based on the movement distance and the movement distance coefficient, a movement distance coefficient determination section that calculates a movement distance coefficient for each section, a movement distance coefficient determination section that calculates the movement distance of the cursor based on the movement distance, and the movement distance coefficient Based on the signal output timing of the extraction signal that is output each time the movement point reaches a preset extraction point, a capture image extraction unit that extracts a capture image from the storage unit, And a capture image storage unit for recording the extracted capture image. The interpolation screen generation part which produces | generates the interpolation screen corresponding to the said signal output timing when the capture image corresponding to the signal output timing of the said extraction signal does not exist in the said preservation | save part is provided. The recording device described.   A storage function for saving a captured image obtained by capturing an operation screen in a storage unit, a display area dividing function for dividing the operation screen into sections, and a cursor indicating a cursor position of the pointing device on the operation screen Cursor movement distance calculation function for calculating the movement distance of the cursor based on position information; movement distance integration function for counting the total movement distance based on the movement distance; and setting a target movement distance of the cursor for each section A target moving distance setting function, and a capture image extracting function for extracting a captured image from the storage unit based on a signal output timing of an extraction signal output every time the integrated moving distance reaches the set target moving distance; A captured image storage function for recording the extracted captured image in a captured image storage unit; Program for the recording apparatus to realize the computer.   A storage function for saving a captured image obtained by capturing an operation screen in a storage unit, a display area dividing function for dividing the operation screen into sections, and a cursor indicating a cursor position of the pointing device on the operation screen Based on the cursor movement distance calculation function that calculates the movement distance of the cursor based on position information, the movement distance coefficient determination function that determines the movement distance coefficient for each section, the movement distance and the movement distance coefficient, Capture image extraction for extracting a capture image from the storage unit based on a movement point calculation function for calculating a movement point and a signal output timing of an extraction signal output each time the movement point reaches a preset extraction point Function and a captured image for recording the extracted captured image in a captured image storage unit Program for the recording apparatus for realizing the pay function, to the computer.

Images