JP2020190663A - Image display system - Google Patents

Abstract

To provide an image display system.SOLUTION: An image display system for displaying huge image information comprises: a first display processing part for displaying image information in a first region forming a part of the huge image information; and a second display processing part for displaying image information in a second region forming all or a part of huge image information and wider than the first region. The second display processing part displays and changes a region corresponding to the checked first region in the second region.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image display system.

With the development of information processing technology in recent years, it may be necessary to check huge image information that greatly exceeds the screen size (vertical and horizontal sizes) of the display device. For example, it is considered to confirm damage such as cracks in social infrastructure structures from huge image information synthesized by photographing social infrastructure structures such as bridges and tunnels with UAVs (Unmanned Aerial Vehicles) and inspection robots. ing. In the fields of pathology and pathological examination, diagnosis may be made from the information of huge images of tissue specimens taken by digital pathology techniques such as digital microscopes and virtual slides.

When displaying such huge image information on a display device, there is a technique as shown in Patent Document 1, for example.

Japanese Unexamined Patent Publication No. 2009-18211

By using the invention described in Patent Document 1, it is possible to display huge image information on a display device while making full use of the resolution. However, since the huge image information cannot be displayed on the display device, the entire huge image information cannot be visually recognized without scrolling vertically and horizontally many times. Therefore, even if the invention described in Patent Document 1 is used, the huge image information can be displayed, but it is not possible to confirm where the operator visually recognizes the huge image information.

For example, when confirming damage to an infrastructure structure from huge image information, the operator needs to confirm whether or not the huge image information is damaged. In that case, the huge image information is checked vertically and horizontally while scrolling appropriately, but it is not easy to check without omission.

Similarly, when making a diagnosis from a huge image of a tissue sample taken with a digital pathology technology such as a digital microscope or virtual slide, the presence or absence of malignant cells in the huge image information is confirmed and the diagnosis is made, but this is also a huge image. It is not easy to confirm all the information.

In view of the above problems, the present inventor has invented an image display system that does not cause omission of confirmation for huge image information.

The first invention is an image display system for displaying huge image information, wherein the image display system performs a display process of image information in a first region which is a part of the huge image information. It has a display processing unit and a second display processing unit that performs display processing of image information in a second area that is all or part of the huge image information and is wider than the first area. The second display processing unit is an image display system that changes the display in the second area of the area corresponding to the first area that has been confirmed.

By configuring as in the present invention, the confirmed partial area can be reflected in the entire area, and the occurrence of confirmation omission can be suppressed.

In the above-described invention, the first display processing unit accepts a predetermined confirmed operation and / or displays the image information of the first area for a predetermined time or longer, so that the first area Can be configured like an image display system that determines that has been confirmed.

Whether or not it has been confirmed may be actively accepted as in the present invention, or may be automatically determined. When actively accepting operations, it is possible to judge whether or not it has been confirmed according to the characteristics of each displayed area, and when automatically judging, it is necessary to prevent forgetting the confirmed operations. Can be done.

In the above-described invention, the second display processing unit is the total area ratio of the confirmed image information of the first region to the image information of the second region or the huge image information. Can be configured like an image display system to display.

By displaying the ratio of the area of the confirmed area, the operator can recognize how much more confirmation should be performed, which can be useful for adjusting the work pace.

In the above-described invention, the image display system determines a priority area in the huge image information, and the first display processing unit and / or the second display processing unit displays the determined priority area. ， Can be configured like an image display system.

Due to the size of huge image information, attention tends to be distracted. Therefore, it is possible to prevent omission of confirmation by displaying the area to be confirmed with particular emphasis as the priority area.

The first invention can be realized by loading the program of the present invention into a computer and executing it. That is, from the first display processing unit that performs the display processing of the image information in the first region that is a part of the huge image information, the computer, all or a part of the huge image information, and the first area. Is an image display program that functions as a second display processing unit that performs display processing of image information in a wide second area, and the second display processing unit is the first area that has been confirmed. This is an image display program that changes the display of the area corresponding to the above in the second area.

By using the image display system of the present invention, it is possible to prevent omission of confirmation of huge image information.

It is a figure which shows typically an example of the whole structure of the image display system of this invention. It is a figure which shows typically an example of the hardware composition of the computer used in the image display system of this invention. It is a flowchart which shows an example of the whole processing process of the image display system of this invention. It is a flowchart which shows an example of the processing process of the display processing of a partial area. It is a flowchart which shows an example of the processing process of the display processing of the whole area. It is a figure which shows an example of a partial area display window. It is a figure which shows an example of the whole area display window. It is a figure which shows typically the transformation of coordinates. It is a figure which shows typically the process of this invention. It is a figure which shows the case where the priority area is displayed in the partial area display window. It is a figure which shows the case where the priority area is displayed in the whole area display window.

FIG. 1 shows an example of the overall system configuration of the image display system 1 of the present invention, and FIG. 2 shows an example of the hardware configuration of the computer used in the image display system 1.

The image display system 1 is realized by a computer. The computer includes an arithmetic unit 70 such as a CPU that executes arithmetic processing of a program, a storage device 71 such as a RAM or a hard disk that stores information, a display device 72 such as a display, and an input device 73 that inputs information. It has a communication device 74 for communicating the processing result of the arithmetic unit 70 and the information stored in the storage device 71. When the computer is provided with a touch panel display, the display device 72 and the input device 73 may be integrally configured. Touch panel displays are often used, for example, in portable communication terminals such as tablet computers and smartphones, but are not limited thereto.

The touch panel display is a device in which the functions of the display device 72 and the input device 73 are integrated in that input can be performed directly on the display with a predetermined input device (such as a pen for a touch panel) or a finger.

The image display system 1 may be realized by one computer, but some or all of its functions may be realized by a plurality of computers. The computer in this case may be, for example, a cloud server.

Each means in the image display system 1 of the present invention is only logically distinguished in its function, and may form the same area physically or substantially.

The input terminal 3 inputs the huge image information to be displayed to the display terminal 2. The huge image information is image information having a vertical and / or horizontal size (number of pixels) larger than the size (number of pixels) of the display area (total area display window 200) for displaying the entire area described later. Is. Preferably, the vertical and / or horizontal size (number of pixels) of the huge image information is larger than the vertical and / or horizontal size (number of pixels) of the display device 72 in the display terminal 2 described later. The input terminal 3 may be, for example, a portable communication terminal such as a tablet computer or a smartphone, or a device for capturing huge image information used in a digital pathology technique such as a digital microscope or a virtual slide.

The display terminal 2 has an image information input reception processing unit 20, an image information storage unit 21, a partial area display processing unit 22, and an entire area display processing unit 23.

The image information input reception processing unit 20 receives input of huge image information from the input terminal 3 and stores it in the image information storage unit 21 described later.

The image information storage unit 21 stores the huge image information input received by the image information input reception processing unit 20.

The partial area display processing unit 22 (first display processing unit) displays the image information 103 of a part of the huge image information (partial area, first area) at a display magnification specified by the operator such as the same magnification. Displayed on the device 72. The partial area display processing unit 22 displays on the display device 72 in the display area (partial area display window 100) for displaying the image information 103 of the partial area. The partial area display window 100 includes an instruction unit 101 for changing the display range such as a scroll bar and an instruction unit for changing the display magnification in order to change the range of the image information 103 of the partial area to be displayed in the window. It is good to have (not shown). FIG. 6 shows an example of the partial area display window 100.

The partial area display processing unit 22 determines whether the operator has confirmed the image information 103 of the partial area displayed in the window by pressing the predetermined confirmed button 102, and sets the display time of the image information 103 of the partial area. When it is determined that the time has been confirmed by measuring the time in the time measuring part (not shown) and displaying the image for a predetermined time or longer, the coordinates of the image information 103 of the partial area displayed in the partial area display window 100 ( By converting the relative coordinates) into the coordinate system of the huge image information, the coordinates (image coordinates) of the partial area in the huge image information are calculated. Then, the calculated image coordinates are passed to the entire area display processing unit 23.

The whole area display processing unit 23 (second display processing unit) displays the image information 203 of the entire area (whole area, second area) of the huge image information on the display device 72. The entire area display processing unit 23 displays the image information 203 of the entire area on the display device 72 in the display area (overall area display window 200) for displaying the image information 203. Since the size of the huge image information is at least larger than that of the whole area display window 200, the whole area display processing unit 23 reduces the display of the huge image information at a magnification that allows the entire huge image information to be displayed in the whole area display window 200. To do. In the present specification, the entire area is preferably the entire huge image information, but it does not necessarily have to be the entire huge image information, but the entire range that needs to be confirmed by the operator, and is a partial area. It suffices if the area is wider than the image information 103 of. FIG. 7 shows an example of the entire area display window 200.

When the whole area display processing unit 23 receives the image coordinates of the partial area displayed and confirmed in the partial area display window 100 from the partial area display processing unit 22, the image coordinates correspond to the image coordinates in the whole area display window 200. Calculate the coordinates (relative coordinates). Then, the process of changing the display is executed with the area of the calculated coordinates as the confirmed area 201. For example, various display change processing such as hatching and changing the color is performed. In FIG. 7, the confirmed area 201 is hatched.

Further, the overall area display processing unit 23 calculates how much of the image information 203 of the entire area has been confirmed, and displays the ratio in the display area (confirmed ratio display area 202). Specifically, the total area of the image information 103 of the confirmed partial area is calculated based on the relative coordinates in the total area display window 200 calculated from the image coordinates of the partial area accepted as confirmed, and the total area is calculated. It is calculated by dividing by the area of the image information of.

The partial area displayed by the partial area display window 100 and the total area displayed by the total area display window 200 are managed by the relative coordinates in each window, and do not match the image coordinate system in the huge image information. Therefore, in order to reflect the area corresponding to the image information 103 of the partial area displayed and confirmed in the partial area display window 100 in the image information 203 of the entire area in the total area display window 200, the coordinate system is converted. There must be. FIG. 8 schematically shows the transformation of the coordinate system.

FIG. 8A is a diagram schematically showing the huge image information, in which the origin of the huge image information in the image coordinate system (absolute coordinates) is O1 (0,0) and the point A is (X, Y). .. FIG. 8B shows image information 103 of a partial region in which a part of the huge image information is displayed at a magnification α, and the origin in the coordinate system (relative coordinates) in the partial region display window 100 is O2 (0,0). Point A is (a, b). Further, FIG. 8C shows image information 203 of the entire area in which the entire huge image information is displayed at a magnification β, and the origin in the coordinate system (relative coordinates) in the entire area display window 200 is O3 (0,0). , Point A is (c, d). The points A correspond to the huge image information, the image information 103 of the partial area, and the image information 203 of the entire area, respectively, but the coordinates do not match because they are all different coordinate systems. Further, the origin O2 of the image information 103 of the partial region is the coordinates of O2 (X0, Y0) in the image coordinate system of the huge image.

The partial area display processing unit 22 needs to convert the coordinates (a, b) of the point A displayed in the partial area into the image coordinate system.
Point A (X, Y) = (a × α + X0, b × α + Y0)
Can be converted to the coordinates of the point A in the image coordinate system by calculating, so the coordinates in the image coordinate system calculated by this calculation are passed to the entire area display processing unit 23.

Further, the whole area display processing unit 23 can convert the point A (X, Y) in the image coordinate system of the huge image information into the coordinates (c, d) of the point A displayed in the whole area display window 200.
Point A (c, d) = (X × β, Y × β)
Is calculated.

In the above description, after converting to the image coordinate system of the huge image, the whole area display processing unit 23 converts the coordinates to the coordinates in the coordinate system, but the relative coordinate system of the partial area display window 100 is converted to the coordinates of the whole area display window 200. You may convert directly to the relative coordinate system. in this case,
Point A (c, d) = ((a × α + X0) × β, (b × α + Y0) × β)
Can be directly converted to the coordinates of the point A displayed in the entire area display window 200 by calculating.

Next, an example of displaying huge image information using the image display system 1 of the present invention will be described. A case will be described in which the huge image information is huge image information obtained by photographing a bridge, and an abnormality such as a crack reflected therein is displayed on the display terminal 2 of the display device 72, and the operator visually confirms the information.

The input terminal 3 inputs the huge image information of the bridge to the display terminal 2, and the image information input reception processing unit 20 of the display terminal 2 accepts the input of the huge image information of the bridge (S100), which is stored in the image information storage unit. It is stored in 21 (S110).

An operator who displays the huge image information obtained by photographing a bridge on the display device 72 of the display terminal 2 and visually confirms the bridge by performing a predetermined operation on the display terminal 2 to request the display of the huge image information. Then, upon receiving this display request (S120), the display terminal 2 extracts the huge image information selected from the image information storage unit 21.

The partial area display processing unit 22 displays the partial area display window 100 on the display device 72 of the display terminal 2, and α = if the magnification α (for example, the same magnification (actual size)) specified in the partial area display window 100. A part of the huge image information is displayed in the size of 1). FIG. 6 is an example of the partial area display window 100 in this case. Then, the partial area display processing unit 22 executes display processing for the partial area display window 100 (S130).

Further, the whole area display processing unit 23 displays the whole area display window 200 on the display device 72 of the display terminal 2 and displays huge image information according to the size of the whole area display window 200. In this case, it is preferable to reduce the entire huge image information and display it, but even if the entire huge image information is not displayed, the huge image in a wider range than the partial area image information 103 displayed in the partial area display window 100 is displayed. A part of the information may be reduced and displayed. Let β be the magnification at which all or part of the huge image information is displayed in the entire area display window 200. Then, the whole area display processing unit 23 executes the display processing for the whole area display window 200 (S140).

Specifically, when the partial area display processing unit 22 receives the designation of the partial area to be displayed by the partial area display processing unit 22 by the operator, the image information 103 of the partial area in the specified range is extracted. Display in the partial area display window 100 (S200).

Then, when the operator confirms the presence or absence of an abnormality in the image information 103 of the partial area, it is determined that the partial area is confirmed by accepting the press of the confirmed button 102 (S210, S220), and the partial area is confirmed. The area display processing unit 22 describes each coordinate (which may be four points or two points diagonally) constituting the confirmed partial area.
X = a × α + X0
Y = b × α + Y0
Is converted into an image coordinate system by calculating (S230).

In addition to accepting the press of the confirmed button 102, the partial area display processing unit 22 determines that the confirmation has been completed by measuring the display of the partial area for a predetermined time with the time measuring unit (not shown). Is also good. Further, both the pressing of the confirmed button 102 and the timekeeping by the timekeeping unit may be used. In this case, if the confirmation is completed early, the next partial area can be confirmed, and at the same time, it is possible to deal with the forgetting to press the confirmed button 102.

When the whole area display processing unit 23 receives the determination that the partial area has been confirmed by the partial area display processing unit 22, it receives each coordinate calculated by the partial area display processing unit 22, and based on the coordinates, the whole area display processing unit 23 receives the coordinates.
c = X × β
d = Y × β
Is calculated to convert to the relative coordinate system in the whole area display processing unit 23 (S300).

Then, in the entire area display window 200, the display change process of the area is performed by each coordinate of the relative coordinate system calculated in S300. In addition, by calculating the area of the area where the display change processing has been performed (the confirmed partial area) and dividing by the area of the entire area, the ratio of the confirmed partial area is converted to the confirmed ratio display area 202. indicate.

When the confirmation of a certain partial area is completed, the operator changes the partial area to be displayed in the partial area display window 100, and performs the confirmation again in the same manner. Then, by similarly executing the display process of the partial area in FIG. 4 and the display process of the entire area in FIG. 5, the changed partial area can be confirmed. FIG. 9 schematically shows such a process.

By executing the above processing, the operator confirms which area is confirmed in the huge image information and which area has been confirmed by visually observing the entire area display window 200. Therefore, it is possible to prevent omission of confirmation.

In the first embodiment, the case where the confirmed partial area in the entire area is displayed and changed has been described, but in the present embodiment, as another embodiment of the image display system 1, the priority area in the entire area may be changed in display. The priority area is an area where the operator should focus on checking the huge image information, for example, an area where the operator needs to check more carefully, or prioritizes before checking the entire huge image information. This is the area to be confirmed.

For example, when the huge image information is the image information of the bridge, the huge image information of the bridge is input to the learning model in which the weighting coefficient between the neurons of each layer of the neural network consisting of many layers is optimized. The damaged area is determined based on the output value. As a learning model in this case, it is possible to use various huge image information of bridges and structures with damaged areas. The area automatically determined by such deep learning or machine learning may be the priority area, or the area determined to be damaged by the past confirmation results may be the priority area. The priority area can be set sequentially for the purpose of displaying huge image information.

As for the area of the part judged to be damaged by the past confirmation result, the huge image information and the position information are associated with each other by the position information acquisition device such as the GPS device when the huge image information is taken. ， It is possible to associate the damaged part in the past huge image information with the position of the area of the current huge image information.

The priority area is the image information 103 of the partial area of the partial area display window 100 displayed by the partial area display processing unit 22, and the image information 203 of the entire area of the total area display window 200 displayed by the total area display processing unit 23, respectively. It can be executed by changing the display. This is schematically shown in FIGS. 10 and 11. FIG. 10 is a diagram showing a case where the priority area 104 is displayed in the partial area display window 100, and FIG. 11 is a diagram showing a case where the priority area 204 is displayed in the entire area display window 200.

When displaying the priority area 204 in the whole area display window 200, the whole area display processing unit 23 may perform different display processing in order to clearly show the difference from the confirmed area 201.

By using the image display system 1 of the present invention, it is possible to prevent omission of confirmation of huge image information.

1: Image display system 2: Display terminal 3: Input terminal 20: Image information input reception processing unit 21: Image information storage unit 22: Partial area display processing unit 23: Overall area display processing unit 70: Arithmetic device 71: Storage device 72 : Display device 73: Input device 74: Communication device 100: Partial area display window 101: Indicator 102: Confirmed button 103: Image information of partial area 104: Priority area 200: Overall area display window 201: Confirmed area 202: Confirmed ratio display area 203: Image information of the entire area 204: Priority area

Claims (5)

An image display system that displays huge image information.
The image display system is
A first display processing unit that performs display processing of image information in a first region that is a part of the huge image information, and
It has a second display processing unit that performs display processing of image information in a second region that is all or part of the huge image information and is wider than the first region.
The second display processing unit is
The display of the area corresponding to the confirmed first area is changed in the second area.
An image display system characterized by this. The first display processing unit is
By accepting a predetermined confirmed operation and / or displaying the image information of the first area for a predetermined time or longer, the first area is judged to be confirmed.
The image display system according to claim 1, wherein the image display system is characterized in that. The second display processing unit is
The total area ratio of the confirmed image information of the first region is displayed with respect to the image information of the second region or the huge image information.
The image display system according to claim 1 or 2, wherein the image display system is characterized in that. The image display system is
Determine the priority area in the huge image information and determine
The first display processing unit and / or the second display processing unit displays the determined priority area.
The image display system according to any one of claims 1 to 3, wherein the image display system is characterized in that. Computer,
A first display processing unit that performs display processing of image information in a first area that is a part of huge image information,
An image display program that functions as a second display processing unit that performs display processing of image information in a second area that is all or part of the huge image information and is wider than the first area.
The second display processing unit is
The display of the area corresponding to the confirmed first area is changed in the second area.
An image display program characterized by this.

Images