WO2022107203A1

WO2022107203A1 - Screen capture device, and screen capture method

Info

Publication number: WO2022107203A1
Application number: PCT/JP2020/042789
Authority: WO
Inventors: 舞坂川; 健太郎金井
Original assignee: 三菱電機株式会社
Priority date: 2020-11-17
Filing date: 2020-11-17
Publication date: 2022-05-27
Also published as: JPWO2022107203A1; JP7520143B2

Abstract

A screen capture device (20) comprises: an image acquisition unit (1); a specification unit (2); an operation processing unit (4); and a processing control unit (5). The image acquisition unit (1) acquires display image data (D1) that is displayed on a display (31). The specification unit (2) specifies the state of a scroll bar on the basis of: the display image data (D1) acquired by the image acquisition unit (1); and scroll image data that is set in advance and that indicates a scroll bar. The operation processing unit (4) operates the scroll bar to scroll display-source data that is displayed by the display (31). The processing control unit (5) controls the operation processing unit (4) on the basis of the specified scroll bar state so as to cause the display (31) to display an undisplayed image area among the display-source data and to cause the image acquisition unit (1) to acquire the display image data (D1) which contains the image area displayed by the display (31).

Description

Screen capture device and screen capture method

This disclosure relates to a screen capture device and a screen capture method.

Conventionally, an automatic test device for performing an operation test has been proposed. The automatic test device displays various information related to the test such as the result information of the operation test on the display. The automatic test device acquires copy data of the image displayed on the display as evidence of the test and records the copy data.

This display may display some information in a window with a scroll bar. In this case, the rest of the information is not shown on the display unless the scroll bar is manipulated. Therefore, it is not possible to acquire the remaining part of the information as evidence only by acquiring the copy data of the image displayed on the display once.

As a technique for solving such a problem, for example, the technique of Patent Document 1 can be used. Patent Document 1 describes a mobile phone including a touch panel. This mobile phone displays a setting image related to printing on a touch panel, and acquires screenshot data (that is, copy data) of the displayed image displayed on the touch panel. When the size of the set image is large, the mobile phone has a scroll bar to display a part of the set image in the display area of the touch panel.

The mobile phone acquires the size of the setting image, and when the size is larger than the size of the display area, it is determined that only a part of the setting image is displayed, and the setting image is scrolled sequentially and displayed on the touch panel. , The copy data of the display image is acquired each time. This makes it possible to acquire all of the set images.

Japanese Unexamined Patent Publication No. 2018-73143

However, in the technique of Patent Document 1, it is necessary to acquire the size of the set image. That is, it is necessary to acquire the size information of the display source data (that is, the setting image) to be displayed on the display. It is necessary to correspond to the data format in order to acquire the size information. Since there are many types of data formats, it takes time and effort to support all data formats.

Therefore, an object of the present disclosure is to provide a technique capable of acquiring a plurality of display screen data including the display source data by a method different from the method of acquiring the size of the display source data. ..

The screen capture device according to the present disclosure includes an image acquisition unit that acquires display image data including a part of display source data and a scroll bar displayed on a display, and the display image acquired by the image acquisition unit. Based on the data and the scroll image data indicating the scroll bar, which is set in advance, the specific unit for specifying the state of the scroll bar and the display source for operating the scroll bar to be displayed on the display. The operation processing unit that scrolls the data and the operation processing unit that controls the operation processing unit based on the state of the scroll bar specified by the specific unit are controlled to display an undisplayed image area of the display source data on the display. It is provided with a processing control unit for causing the image acquisition unit to acquire the display image data including the image area displayed on the display.

The screen capture method according to the present disclosure acquires display image data displayed on a display, obtains the displayed image data, and sets in advance scroll image data indicating a scroll bar displayed on the display. Based on this, the state of the scroll bar is specified, and the scroll bar is operated based on the specified state of the scroll bar to display an undisplayed image area of the display source data on the display. The display image data including the image area displayed on the display is acquired.

According to the present disclosure, it is possible to acquire a plurality of display screen data including the display source data by a method different from the method of acquiring the size of the display source data.

The purposes, features, aspects, and advantages of the present disclosure will be made clearer by the following detailed description and accompanying drawings.

It is a block diagram which shows typically an example of the structure of the screen capture apparatus which concerns on embodiment. It is a figure which shows the example of the procedure data which shows the procedure of the operation test schematically. It is a figure which shows the example of the display image displayed on the display schematically. It is a figure which shows an example of the relationship between the position of a scroll box and the position of an image area displayed on a display in the display source data schematically. It is a figure which shows an example of the relationship between the position of a scroll box and the position of an image area displayed on a display in the display source data schematically. It is a figure which shows an example of the relationship between the position of a scroll box and the position of an image area displayed on a display in the display source data schematically. It is a figure which shows the example of the 1st scroll image data schematically. It is a figure which shows the example of the 1st scroll image data schematically. It is a figure which shows the example of the 2nd scroll image data schematically. It is a flowchart which shows an example of the operation of a screen capture device. It is a figure which shows the example of the hardware composition of the screen capture apparatus schematically. It is a figure which shows the example of the hardware composition of the screen capture apparatus schematically.

Hereinafter, embodiments will be described with reference to the attached drawings. In the following embodiments, detailed features and the like are also shown for illustration purposes, but they are exemplary and not all of them are essential features for the embodiments to be feasible.

It should be noted that the drawings are shown schematically, and for convenience of explanation, the configuration is omitted or the configuration is simplified as appropriate in the drawings. Further, the interrelationship between the sizes and positions of the configurations and the like shown in different drawings is not always accurately described and can be changed as appropriate. Further, even in a drawing such as a plan view which is not a sectional view, hatching may be added to facilitate understanding of the contents of the embodiment.

Further, in the explanation shown below, similar components are illustrated with the same reference numerals, and their names and functions are the same. Therefore, detailed description of them may be omitted to avoid duplication.

Further, in the description described below, when it is described that a certain component is "equipped", "included", or "has", the existence of another component is excluded unless otherwise specified. Not an expression.

Also, even if ordinal numbers such as "first" or "second" may be used in the description described below, these terms facilitate the understanding of the content of the embodiments. It is used for convenience, and is not limited to the order that can be generated by these ordinal numbers.

Further, in the description described below, expressions indicating equality, such as "same", "equal", "uniform" or "homogeneous", are strictly equal unless otherwise specified. It shall include the case where it indicates that there is, and the case where there is a difference within the range where tolerance or similar function can be obtained.

FIG. 1 is a block diagram schematically showing an example of the configuration of the screen capture device 20 according to the embodiment. The screen capture device 20 is a device for capturing a display image displayed on the display screen 31a of the display 31. In other words, the screen capture device 20 is a device for acquiring copy data of the display image data D1 indicating the display image displayed on the display screen 31a. It can be said that this copy data is screenshot data. Since the copy data of the display image data D1 is substantially the same as the display image data D1, the copy data may also be simply referred to as the display image data D1 in the following.

In the example of FIG. 1, the screen capture device 20 is electrically connected to a computer device 30 such as a personal computer. For example, the computer device 30 includes a display 31 such as a liquid crystal display and a computer main body 32. The computer main body 32 outputs the display image data D1 to the display 31, and the display 31 displays the display image data D1 on the display screen 31a. The display image data D1 is, for example, data including pixel values (for example, luminance values) for each pixel constituting the display screen 31a of the display 31. By displaying the display image data D1 on the display screen 31a, the display 31 displays various information such as characters, symbols, and images contained in the display image data D1 on the display screen 31a of the display 31.

The computer body 32 has a function of performing various arithmetic processes. For example, the computer body 32 performs processing in response to input to an input device (for example, a mouse and a keyboard) by a user. The computer main body 32 can display the result of the processing on the display 31. For example, the computer main body 32 generates display image data D1 that reflects the result of processing, and outputs the display image data D1 to the display 31. As a result, the display image data D1 reflecting the processing is displayed on the display screen 31a of the display 31. Further, the computer main body 32 also has a function of storing copy data of the display image data D1 displayed on the display screen 31a.

In the example of FIG. 1, the screen capture device 20 is also electrically connected to the automatic test device 40. The automatic test device 40 is a device for automatically performing an operation test. The test referred to here is, for example, a test for determining the suitability of the operation of a predetermined component. A given component may include software. The software may be installed, for example, in the computer body 32 and executed by the computer body 32.

Procedure data 50 showing the test procedure is input to the automatic test device 40. For example, the procedure data 50 is a script that defines the procedure of the test, and may be specified in a predetermined programming language. The automatic test device 40 automatically performs a test based on the input procedure data 50.

FIG. 2 is a diagram schematically showing an example of the contents of the procedure data 50. In the example of FIG. 2, the procedure data 50 defines the

test procedures

51 and 52. Here, as an example, the software executed by the computer main body 32 is the subject of the test. In step 51, the click of the button A, the click of the button B, and the screen capture are defined in this order from the upper side to the lower side. Buttons A and B are software keys displayed on the display screen 31a when the computer main body 32 executes software. When the button A is clicked, the display content of the display screen 31a changes according to the button A, and when the button B is clicked, the display content of the display screen 31a changes according to the button B.

The automatic test device 40 executes each step specified in the procedure data 50 in order from the upper side. Specifically, the automatic test device 40 instructs the computer main body 32 to click the buttons A and B in this order. The automatic test device 40 may output an instruction to the computer main body 32 via the screen capture device 20, or may directly output an instruction to the computer main body 32.

The computer body 32 clicks button A and then clicks button B in response to the instruction. That is, the computer main body 32 executes the process corresponding to the button A and the process corresponding to the button B in this order, and sequentially generates the display image data D1 reflecting the result. The computer main body 32 displays the display image data D1 on the display 31 in order. Since the latest display image data D1 includes the results of both the clicks of the buttons A and B, the display screen 31a of the display 31 displays the processing results for the clicks of the buttons A and B.

The automatic test device 40 instructs the screen capture device 20 to capture the screen after instructing the buttons A and B to click. The screen capture device 20 performs a process for capturing the display image of the display screen 31a in response to the instruction. For example, the screen capture device 20 instructs the computer main body 32 to capture the screen. In response to the instruction, the computer main body 32 stores the copy data of the latest display image data D1 after clicking the buttons A and B. Alternatively, the screen capture device 20 may acquire the latest copy data of the display image data D1 from the computer main body 32 and store the copy data.

According to the test according to the procedure 51 as described above, the display image data D1 including the processing results for the clicks of the buttons A and B can be stored as evidence.

In step 52, the keyboard input of "test", the click for the "OK" button, and the screen capture are specified in this order from the upper side to the lower side. The "OK" button is a software key displayed on the display screen 31a when the computer main body 32 executes the software. When the "OK" button is clicked, the display content of the display screen 31a changes according to the "OK" button.

When the automatic test device 40 executes the procedure 52 specified in the procedure data 50, the computer main body 32 performs the processing corresponding to the character input of the "test", and then the processing corresponding to the click of the "OK" button. I do. The computer main body 32 sequentially generates display image data D1 reflecting these processing results, and sequentially outputs the display image data D1 to the display 31. The display 31 displays the display image data D1 on the display screen 31a in order.

The automatic test device 40 instructs the screen capture device 20 to capture the screen after the keyboard input of "test" and the instruction of clicking the "OK" button. The screen capture device 20 performs a process for capturing the display image of the display screen 31a in response to the instruction. For example, the screen capture device 20 instructs the computer main body 32 to capture the screen. In response to the instruction, the computer main body 32 stores the latest copy data of the display image data D1 after the keyboard input of "test" and the click to the "OK" button are performed. Alternatively, the screen capture device 20 acquires the latest copy data of the display image data D1 from the computer main body 32 and stores the copy data.

According to the test according to the procedure 52, the latest display image data D1 includes the processing result for the keyboard input of "test" and the click of the "OK" button. Therefore, the display image data D1 including the processing result for the keyboard input of the "test" and the click of the "OK" button can be stored as evidence.

A window with a scroll bar may be displayed on the display screen 31a of the display 31 as described later. In this case, only a part of the information of the display source is displayed in the window, and the remaining part of the information is displayed by scrolling the image in the window. Therefore, in this case, it is necessary to capture the displayed image while scrolling the image in order to properly acquire the necessary information.

In the example of FIG. 1, the screen capture device 20 includes an image acquisition unit 1, a specific unit 2, an image position acquisition unit 3, an operation processing unit 4, and a processing control unit 5.

The image acquisition unit 1 acquires the display image data D1 from the computer main body 32. The image acquisition unit 1 may store the display image data D1 in a non-temporary non-volatile storage unit. It can be said that the image acquisition unit 1 captures the display image displayed on the display screen 31a of the display 31.

FIG. 3 is a diagram schematically showing an example of a display image displayed on the display screen 31a of the display 31. In the example of FIG. 3, the display image includes the first window 34 and the second window 35. In the example of FIG. 3, the scroll bar is not attached to the first window 34, and the scroll bar 33 is attached to the second window 35. In the following, the second window 35 will be mainly focused on.

The scroll bar 33 includes an element 331 indicating its upper end (corresponding to the first end), an element 332 indicating its lower end (corresponding to the second end), and a scroll box 333. The scroll box 333 can be moved vertically between the upper and lower ends of the scroll bar 33. The scroll box 333 may also be referred to as a knob, valve, thumb, crawler or scroll thumb.

The display 31 displays a part of the display source data D2 on the display screen 31a together with the scroll bar 33. Here, the display source data D2 and the position of the scroll box 333 on the scroll bar 33 will be described. 4 to 6 are diagrams schematically showing an example of the relationship between the position of the scroll box 333 and the position of the image area (here, the second window 35) displayed on the display 31 of the display source data D2. Is.

In the example of FIG. 4, the scroll box 333 is located at the upper end position of the scroll bar 33. At this time, the uppermost area of the display source data D2 is displayed in the second window 35. In the example of FIG. 5, the scroll box 333 is located at an intermediate position between the upper end and the lower end of the scroll bar 33. Therefore, at this time, the intermediate region of the display source data D2 is displayed in the second window 35. In the example of FIG. 6, the scroll box 333 is located at the lower end position of the scroll bar 33. Therefore, at this time, the lowermost region of the display source data D2 is displayed in the second window 35.

As described above, the position of the scroll box 333 in the scroll bar 33 indicates the image area displayed in the second window 35 of the display source data D2. Conversely, if the state of the scroll bar 33 can be specified, the image area displayed in the second window 35 of the display source data D2 can be specified. Here, the state of the scroll bar 33 may include the position and size of the scroll box 333 in the scroll bar 33.

Therefore, in order to specify the state of the scroll bar 33 displayed on the display screen 31a, the scroll image data indicating the scroll bar 33 is stored in the storage unit 10 in advance. The storage unit 10 is, for example, a non-volatile storage unit, and as a more specific example, it is a hard disk drive (Hard disk drive, that is, HDD), a read-only memory (that is, ROM), a flash memory, and an erase program. Memory (storage medium) including non-volatile semiconductor memory, magnetic disk, flexible disk, optical disk, compact disk, mini disk, DVD, etc. such as read only memory (EPROM) and electrically manageable program-only memory (EEPROM), Alternatively, it may be any storage medium used in the future.

The scroll image data includes, for example, the first scroll image data D31 indicating the scroll bar 33 in a state where the scroll box 333 is not located at the lower end of the scroll bar 33, and the scroll box 333 is located at the lower end of the scroll bar 33. The second scroll image data D32 indicating the scroll bar 33 of the above is preset. That is, the first scroll image data D31 and the second scroll image data D32 are stored in the storage unit 10 in advance.

7 and 8 are diagrams schematically showing an example of the first scroll image data D31, and FIG. 9 is a diagram schematically showing an example of the second scroll image data D32. The first scroll image data D31 and the second scroll image data D32 are, for example, data obtained by cutting out a part of the display image data D1 and correspond to a lower end region of the scroll bar 33 (corresponding to a partial region on the second end side). It is image data which shows. Both the first scroll image data D31 and the second scroll image data D32 include a part above the element 332 and the element 332. The first scroll image data D31 in FIG. 7 does not include the scroll box 333. Although the first scroll image data D31 of FIG. 8 includes the lower end portion of the scroll box 333, there is a gap between the scroll box 333 and the element 332, and the scroll box 333 is larger than the lower end portion of the scroll bar 33. It is located on the upper side. As the first scroll image data D13, more image data in which the positions of the scroll boxes 333 are different may be adopted. The second scroll image data D32 in FIG. 9 includes a scroll box 333 located at the lower end of the scroll bar 33.

The specifying unit 2 specifies the state of the scroll bar 33 in the display image data D1 acquired by the image acquisition unit 1 based on the scroll image data stored in the storage unit 10 and the display image data D1. Specifically, the specific unit 2 acquires the first scroll image data D31 and the second scroll image data D32 from the storage unit 10. Then, the specific unit 2 determines whether or not the first scroll image data D31 or the second scroll image data D32 can be detected from the display image data D1.

For example, the specific unit 2 performs template matching between the display image data D1 and the first scroll image data D31. As illustrated in FIGS. 7 and 8, when a plurality of types of the first scroll image data D31 exist, template matching may be performed for each of the first scroll image data D31. When the display image data D1 contains a first image similar to or matching each first scroll image data D31, the first image is detected by the template matching. In this case, in the scroll bar 33 displayed on the display screen 31a, the scroll box 333 is located above the lower end position.

The specific unit 2 also performs template matching between the display image data D1 and the second scroll image data D32. When the display image data D1 contains a second image similar to or matching the second scroll image data D32, the second image is detected by the template matching. In this case, in the scroll bar 33 displayed on the display screen 31a, the scroll box 333 is located at the lower end position.

The image position acquisition unit 3 outputs the position information of the scroll box 333 of the scroll bar 33 to the processing control unit 5. For example, when a first image similar to or matching the first scroll image data D31 is detected by the specific unit 2, the image position acquisition unit 3 tells the processing control unit 5 that the scroll box 333 is at a position other than the lower end position. Notice. Further, when the specific unit 2 detects a second image similar to or matching the second scroll image data D32, the image position acquisition unit 3 notifies the processing control unit 5 that the scroll box 333 is at the lower end position. Further, when neither the first image nor the second image is detected from the display image data D1, the image position acquisition unit 3 may notify the processing control unit 5 to that effect.

The operation processing unit 4 performs processing related to the operation of the scroll bar 33. The operation processing unit 4 operates the scroll bar 33 to scroll the display source data D2 to be displayed on the display 31. For example, the operation processing unit 4 instructs the computer main body 32 to operate the scroll bar 33. Specifically, the operation processing unit 4 lowers the scroll box 333 so as to display the image area one page or less below the currently displayed image area of the display source data D2. The computer main body 32 is instructed to move to. The operation of moving the image area downward by one page is also called a page down operation.

The operation processing unit 4 may further perform processing related to the screen capture operation. For example, the operation processing unit 4 may instruct the computer main body 32 to perform a screen capture operation. In response to the instruction, the computer main body 32 stores the copy data of the display image data D1 displayed on the display 31 in, for example, a non-temporary non-volatile storage unit.

The processing control unit 5 comprehensively processes the entire screen capture device 20. For example, the processing control unit 5 outputs various instructions to the image acquisition unit 1, the specific unit 2, the image position acquisition unit 3, and the operation processing unit 4, and the image acquisition unit 1, the specific unit 2, and the image position acquisition unit 4. Various information is received from 3 and the operation processing unit 4.

FIG. 10 is a flowchart showing an example of the operation of the screen capture device 20. In step S1, the processing control unit 5 determines whether or not the screen capture instruction from the automatic test device 40 has been received. When the instruction is not received, the processing control unit 5 executes step S1 again.

When receiving the screen capture instruction, in step S2, the processing control unit 5 causes the image acquisition unit 1 to acquire the display image data D1. The image acquisition unit 1 acquires the display image data D1 from the computer main body 32. Further, when the computer main body 32 is to perform the screen capture, the processing control unit 5 causes the operation processing unit 4 to instruct the computer main body 32 to perform the screen capture operation. In response to the instruction, the computer main body 32 stores the copy data of the display image data D1 in the non-temporary non-volatile storage unit as evidence. When the screen capture device 20 performs screen capture, the image acquisition unit 1 may store the display image data D1 in the storage unit. As a result, in step S2, the currently displayed display image data D1 is stored in the storage unit as evidence.

Next, in step S3, the processing control unit 5 causes the specific unit 2 to determine whether or not a first image similar to or matching the first scroll image data D31 is detected from the display image data D1. Specifically, the specific unit 2 performs template matching between the display image data D1 acquired by the image acquisition unit 1 and the first scroll image data D31 stored in the storage unit 10. When a plurality of types of first scroll image data D31 are stored in the storage unit 10, the specific unit 2 performs template matching for each of the first scroll image data D31.

When any of the first images is detected, in step S3, the processing control unit 5 causes the operation processing unit 4 to operate the scroll bar 33, and displays the undisplayed image area of the display source data D2 with the display 31. It is displayed in the second window 35 of. For example, the operation processing unit 4 moves the scroll box 333 to the lower end side on one page or less, and displays an undisplayed image area on the second window 35 of the display 31. As a more specific example, the operation processing unit 4 instructs the computer main body 32 to perform a page-down operation. The computer main body 32 performs a page down operation based on the instruction. As a result, the image area of the display source data D2 that is one page lower than the image area displayed in the second window 35 is displayed in the second window 35.

The page-down operation can be realized, for example, by clicking a part of the area between the scroll box 333 and the element 332 in the scroll bar 33 (see, for example, the position P1 in FIG. 7). The click position P1 is, for example, a position above the element 332 in the first scroll image data D31 (first image). The image position acquisition unit 3 may specify the click position P1 based on the position of the first image detected by the identification unit 2 (see FIGS. 7 and 8), and the operation processing unit 4 may acquire the image position. The position P1 specified by the unit 3 may be specified, and the click may be instructed to the computer main body 32. The computer main body 32 performs a page down operation by executing a click on the position P1 in response to the instruction.

Next, the processing control unit 5 executes step S2 again. That is, the undisplayed image area of the display source data D2 is displayed in the second window 35 by the scroll operation in step S4. Therefore, in order to acquire the displayed contents, the image acquisition unit 1 is displayed in step S2. Acquires the display image data D1 displayed on the display screen 31a. The display image data D1 displayed in step S2 is stored in the non-volatile storage unit as evidence by the computer main body 32 or the image acquisition unit 1.

The set of steps S4 and S2 is repeatedly executed as long as the first image is detected from the display image data D1 in step S3. That is, when the first image is detected, the display source data D2 includes an undisplayed image area. Therefore, the operation processing unit 4 performs a scroll operation to display the undisplayed image area on the display 31. It is displayed on the display screen 31a (step S4). Then, the display image data D1 including the image area displayed on the display screen 31a is acquired and stored (step S2). As a result, the image area of the display source data D2 that has not been acquired yet can be sequentially acquired as the display image data D1 and stored in the storage unit.

On the other hand, when the first image is not detected from the display image data D1 in step S3, the processing control unit 5 determines in step S5 whether or not the second image is detected from the display image data D1. Let the specific part 2 do it. Specifically, the specific unit 2 performs template matching between the display image data D1 acquired by the image acquisition unit 1 and the second scroll image data D32 stored in the storage unit 10.

When the second image is detected from the display image data D1, the scroll box 333 is located at the lower end position, so the process ends. That is, when the scroll box 333 is located at the lower end position, the image area of the data D2 is sequentially displayed on the display screen 31a by the previous scroll operation (step S4), and the display image data D1 is acquired for each scroll operation. It is stored in the non-volatile storage unit. As a result, the entire image area of the display source data D2 is divided into a plurality of display image data D1 and acquired. In other words, the processing is terminated because sufficient evidence has been obtained.

When the second image is not detected from the display image data D1 in step S5, the processing control unit 5 determines that an abnormality has occurred in step S6 and performs error processing. That is, when neither the first image nor the second image can be detected from the display image data D1, the processing control unit 5 determines that an abnormality has occurred. As error processing, the processing control unit 5 may display, for example, on the display 31 that the state of the scroll bar 33 could not be specified.

As described above, in the present embodiment, the state of the scroll bar 33 is specified based on the display image data D1 and the preset scroll image data, and the scroll operation and the scroll operation are performed based on the state of the scroll bar 33. Capture the display screen. According to such a process, it is not necessary to acquire the size information of the display source data D2. In other words, the necessity of the scroll operation can be determined by a method different from the method of acquiring the size information of the display source data D2, and by extension, the entire image area of the display source data D2 is displayed as a plurality of display images. It can be acquired by dividing it into data D1. Therefore, the screen capture device 20 does not need to perform processing corresponding to the data format of the data D2, and the introduction of the screen capture device 20 becomes easy.

Further, in the above example, every time it is determined that scrolling is necessary, the scroll box 333 is moved from the upper end to the lower end by one page or less (step S4), and the display image is captured each time. Therefore, according to the present embodiment, each image area of the display source data D2 can be captured in order from the upper side. In other words, each image area can be acquired as display image data D1 in order from the upper side.

According to such a scroll order, when the scroll box 333 is not yet located at the lower end position, the display source data D2 includes an unacquired image area.

Therefore, the screen capture device 20 performs template matching between the first scroll image data D31 indicating the lower end region of the scroll bar 33 not including the scroll box 333 and the display image data D1. According to this, the screen capture device 20 can easily identify the state in which the scroll box 333 is not located at the lower end position. In other words, the screen capture device 20 can easily identify that the image area that has not yet been acquired as evidence is not included in the display source data D2 and the scroll operation is necessary.

Further, according to the scroll order described above, when the scroll box 333 is located at the lower end position, the entire image area of the display source data D2 is captured.

Therefore, the screen capture device 20 performs template matching between the second scroll image data D32 indicating the scroll bar 33 in a state where the scroll box 333 is located at the lower end position and the display image data D1. According to this, the screen capture device 20 can easily identify the state in which all of each image area of the display source data D2 is acquired as evidence.

Further, in the above example, when the processing control unit 5 cannot detect both the first image and the second image from the display image data D1 by template matching, it is determined that an abnormality has occurred. Therefore, the processing control unit 5 can perform error processing at an appropriate timing. As error processing, the processing control unit 5 displays, for example, that an abnormality has occurred on the display 31.

In the above example, although the scroll bar 33 is a vertical scroll bar, it may be a horizontal scroll bar.

Next, an example of the hardware configuration of the screen capture device 20 will be described. 11 and 12 are diagrams schematically showing an example of the hardware configuration of the screen capture device 20 exemplified in FIG. 1.

The hardware configurations exemplified in FIGS. 11 and 12 may not match the numbers of the configurations exemplified in FIG. 1, but the configuration exemplified in FIG. 1 is a conceptual unit. It is due to the fact that it is shown.

In FIG. 11, a process for performing an operation as a hardware configuration for realizing an image acquisition unit 1, a specific unit 2, an image position acquisition unit 3, an operation processing unit 4, and a processing control unit 5 in FIG. The circuit 21A and the storage device 22 are shown.

The storage device 22 is, for example, a hard disk drive (Hard disk drive, that is, HDD), a random access memory (random access memory, that is, RAM), a read-only memory (read only memory, that is, ROM), a flash memory, and an erase program. Memory (storage) including volatile or non-volatile semiconductor memory, magnetic disk, flexible disk, optical disk, compact disk, mini disk or DVD such as read only memory (EPROM) and electrically rememberable program-only memory (EEPROM). It may be a medium) or any storage medium that will be used in the future.

The processing circuit 21A may execute a program stored in a storage device 22, an external CD-ROM, an external DVD-ROM, an external flash memory, or the like. That is, for example, it may be a central processing unit (CPU), a microprocessor, a microcomputer, or a digital signal processor (DSP).

When the processing circuit 21A executes a program stored in a storage device 22, an external CD-ROM, an external DVD-ROM, an external flash memory, or the like, the image acquisition unit 1 and the specific unit 2 The image position acquisition unit 3, the operation processing unit 4, and the processing control unit 5 are realized by software, firmware, or a combination of software and firmware in which a program stored in the storage device 22 is executed by the processing circuit 21A. .. The functions of the image acquisition unit 1, the specific unit 2, the image position acquisition unit 3, the operation processing unit 4, and the processing control unit 5 may be realized, for example, by linking a plurality of processing circuits. ..

The software and firmware may be described as a program and stored in the storage device 22. In that case, the processing circuit 21A realizes the above function by reading and executing the program stored in the storage device 22. That is, the storage device 22 may store a program in which the above functions are eventually realized by being executed by the processing circuit 21A.

Further, the processing circuit 21B may be dedicated hardware. That is, the processing circuit 21B is, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an integrated circuit (application specific integrated circuit, that is, an ASIC), a field-programmable gate array (FPGA). Alternatively, the circuit may be a combination of these.

When the processing circuit 21B is dedicated hardware, the image acquisition unit 1, the specific unit 2, the image position acquisition unit 3, the operation processing unit 4, and the processing control unit 5 are realized by operating the processing circuit 21B. The functions of the image acquisition unit 1, the specific unit 2, the image position acquisition unit 3, the operation processing unit 4, and the processing control unit 5 may be realized by separate circuits or may be realized by a single circuit. good.

The functions of the image acquisition unit 1, the specific unit 2, the image position acquisition unit 3, the operation processing unit 4, and the processing control unit 5 are for executing a program partially stored in the storage device 22. It may be realized in the processing circuit 21B, which is a part of the dedicated hardware.

Further, the functions of the computer main body 32 and the functions of the automatic test device 40 may be realized by the processing circuit 21A and the storage device 22 or may be realized by the processing circuit 21B, similarly to the screen capture device 20.

As described above, the present disclosure has been explained in detail, but the above description is exemplary in all aspects and is not limiting. It is understood that innumerable variations not illustrated can be assumed.

Further, as long as there is no contradiction, the components described as being provided with "one" in the above-described embodiment may be provided with "one or more".

Furthermore, each component in the embodiments described above is a conceptual unit, and within the scope of the art disclosed herein, one component comprises a plurality of structures. It is assumed that one component corresponds to a part of a structure, and further, a case where a plurality of components are provided in one structure is included.

Further, each component in the above-described embodiment shall include a structure having another structure or shape as long as it exhibits the same function.

Further, the description in the present specification is referred to for all purposes related to the present art, and none of them is recognized as the prior art.

Further, each component described in the above-described embodiment is assumed to be software or firmware and corresponding hardware, and in both concepts, each component is a "part". Alternatively, it is referred to as a "processing circuit" or the like.

Further, the technique disclosed in the present specification may be provided in a case where each component is distributed and provided in a plurality of devices, that is, a mode such as a system may be used.

1 image acquisition unit, 2 specific unit, 31 display, 33 scroll bar, 333 scroll box, 4 operation processing unit, 5 processing control unit, D1 display image data, D31 1st scroll image data, D32 2nd scroll image data.

Claims

An image acquisition unit that acquires display image data including a part of the display source data and a scroll bar displayed on the display.
A specific unit that specifies the state of the scroll bar based on the display image data acquired by the image acquisition unit and preset scroll image data indicating the scroll bar.
An operation processing unit that operates the scroll bar to scroll the data of the display source to be displayed on the display, and an operation processing unit.
The operation processing unit is controlled based on the state of the scroll bar specified by the specific unit to display an undisplayed image area of the display source data on the display, and the display is displayed on the display. A screen capture device including a processing control unit that causes the image acquisition unit to acquire the display image data including an image area.
The scroll box of the scroll bar moves between the first and second ends of the scroll bar.
As the scroll image data, first scroll image data indicating the scroll bar in a state where the scroll box is not located at the second end of the scroll bar is set in advance.
The processing control unit
When a first image similar to or matching the first scroll image data is detected from the display image data, the operation processing unit is controlled to move the scroll box to the second end side in one page or less. The screen capture device according to claim 1, wherein the undisplayed image area is displayed on the display.
As the scroll image data, second scroll image data indicating the scroll bar in a state where the scroll box is located at the second end of the scroll bar is set in advance.
The screen capture device according to claim 2, wherein the processing control unit ends processing when a second image similar to or matching the second scroll image data is detected from the displayed image data.
The screen capture device according to claim 3, wherein the processing control unit determines that an abnormality has occurred when neither the first image nor the second image is detected from the displayed image data.
Acquire the display image data displayed on the display and
The state of the scroll bar is specified based on the acquired display image data and the preset scroll image data indicating the scroll bar displayed on the display.
The scroll bar is operated based on the specified state of the scroll bar to display an undisplayed image area of the display source data on the display.
A screen capture method for acquiring the displayed image data including the image area displayed on the display.