JP2023045140A - Information processing device, information processing method, and program - Google Patents

Abstract

SOLUTION: A control device according to the present invention comprises: a display control unit for controlling a display apparatus; a peripheral apparatus detection unit for detecting one or more peripheral apparatuses existing in the surrounding of a display device and having an optical shutter and/or an imaging element; and a determination unit for determining whether or not a suspicious apparatus is included among the one or more peripheral apparatuses detected by the peripheral apparatus detection unit that does not conform to a genuine product criterion that pertains to apparatuses permitted to conduct a process of rendering secret information recognizable. The display control unit is constituted to be able to execute: (a) a first concealment process of causing the display apparatus to display a secret image for displaying secret information and an obstacle image for obstructing at least a portion of the secret image from being visually recognizable separately in time when it is not determined by the determination unit that a suspicious apparatus is included among the one or more peripheral apparatuses; and (b) a second concealment process of concealing at least a portion of secret information in a different manner than in the first concealment process when it is determined by the determination unit that a suspicious apparatus is included among the one or more peripheral apparatuses.SELECTED DRAWING: Figure 1

Description

The present invention relates to an information processing device, an information processing method, and a program.

Patent Documents 1 to 7 disclose, for example, generation of an opposite color image for each frame to cancel a display image of one frame, time division of the display image of one frame into display images of a plurality of subfields, and An image display device that time-divides a color image into a plurality of sub-fields of opposite color images and arranges one sub-field of opposite color images before and after the display time of one sub-field display image to generate a time-divided image. is disclosed.
[Prior art documents]
[Patent Literature]
[Patent Document 1] JP-A-2009-204948 [Patent Document 2] JP-A-06-118927 [Patent Document 3] Patent No. 5207175 [Patent Document 4] Patent No. 5447391 [Patent Document 5] Patent No. 5867117 [Patent Document 6] International Publication No. 2012/120853 [Patent Document 7] Patent No. 6145622

In a first aspect of the invention, a controller is provided. The control device described above controls, for example, a display device having a display device for displaying an image. The control device described above includes, for example, a display control unit that controls the display device. The control device described above includes, for example, at least one of an optical shutter and an imaging device, and a peripheral device detection unit that detects one or more peripheral devices existing around the display device. In the above control device, for example, among the one or more peripheral devices detected by the peripheral device detection unit, a device that does not meet the genuine product determination condition, which is the condition regarding the device for which the confidential information recognition enablement process is permitted. A determination unit is provided for determining whether or not a suspicious device is included. In the above control device, the display control unit, for example, (a) when the determination unit does not determine that the suspicious device is included in the one or more peripheral devices, a confidential image for displaying confidential information; It is configured to be able to execute a first anonymization process of displaying an obstructive image for obstructing visual recognition of at least part of the confidential image on the display device in a time division manner. In the control device described above, for example, when the (b) determination unit determines that the suspicious device is included in the one or more peripheral devices, the display control unit displays the confidential information in a manner different from the first anonymization process. It is configured to be able to execute a second anonymizing process for anonymizing at least part of the information. In the control device described above, the second anonymization processing includes processing for stopping display of the confidential image on the display device, processing for changing the timing at which the confidential image is displayed on the display device, and processing for displaying the confidential information. processing to reduce the window that is displayed, processing to move the window behind other windows, processing to change the display position of the window, and processing to change the color tone, contrast or brightness of the confidential image including at least one of

In the control device described above, the processing for changing the timing at which the confidential image is displayed on the display device includes the processing for changing the pattern for defining display and non-display of the confidential image, and changing the start time of the pattern. and at least one of processing for changing the period of the pattern. In the above control device, the interfering image includes, among the confidential images, an image for obstructing the visual recognition of the image displayed on substantially the entire screen display area of the display device, An image for obstructing the visual recognition of the image displayed substantially in the center, an image for obstructing the visual recognition of the image displayed on the specific scanning line of the display device among the hidden images, and the hidden image, specified at least one of the images for obstructing the viewing of images having a color tone or brightness of .

The control device described above may comprise a transmission control section that controls a transmitter configured to communicate with at least one of the one or more peripheral devices. In the above control device, when the display control unit determines to change the timing at which the confidential image is displayed, the transmission control unit selects the optical shutter among the one or more peripheral devices detected by the peripheral device detection unit. The transmitter may be controlled so that information indicating a change in the timing at which the confidential image is displayed is transmitted to the authorized device, which is a device that has the above and satisfies the genuine product determination condition. In the control device described above, the transmitter may be configured to be capable of wireless communication with the authorized device. In the above-described control device, the transmission control unit, when the display control unit determines to change the timing of displaying the confidential image, transmits radio waves for transmitting information indicating a change in the timing of displaying the confidential image. The transmitter may be controlled such that the radio field strength is weaker than a predetermined strength.

In the control device described above, the image displayed on the display device may include one or more display windows for displaying information presented to the user. In the control device described above, the confidential image may include at least an image of the image display area of the display window for displaying confidential information. In the control device described above, the display control unit may arrange a full-screen window covering substantially the entire screen display area of the display device in front of the one or more display windows. In the control device described above, the display control unit may control the display device so that the confidential image and the disturbing image for each frame are displayed in the full-screen window in a time division manner.

The control device described above may include a secret image generation unit that generates a secret image. The control device described above may comprise a disturbance image generator for generating a disturbance image. The control device described above may include a target specifying unit that specifies a target window to be subjected to anonymization processing among the one or more display windows. The control device described above may include a window information acquiring section that acquires window information indicating the position, shape, and size of the target window specified by the target specifying section. In the control device described above, the interference image generation unit generates the position, shape, and size at which the interference image is displayed on the display device based on the position, shape, and size of the target window indicated by the window information acquired by the window information acquisition unit. size can be determined. In the control device described above, the disturbing image generator may generate a disturbing window for displaying the disturbing image. In the control device described above, the display control unit (i) places the target window in front of the disturbance window when the confidential image is displayed on the display device, and (ii) displays the disturbance image on the display device. If so, the display device may be controlled so that the disturbing window is placed in front of the target window.

In a second aspect of the invention, a display device is provided. The display device described above includes, for example, a display device that displays an image. The display device described above includes, for example, a control device that controls the display device. Said control apparatus is provided with the control apparatus which concerns on said 1st aspect, for example.

In a third aspect of the invention, a control method is provided. The control method described above is, for example, a control method for controlling a display device having a display device for displaying an image. The control method described above comprises, for example, a display control stage for controlling the display device. The above control method has, for example, at least one of an optical shutter and an imaging device, and has a peripheral device detection step of detecting one or more peripheral devices existing around the display device. In the above control method, for example, among the one or more peripheral devices detected in the peripheral device detection stage, a device that does not meet the genuine product determination condition, which is a condition regarding the device for which confidential information recognition processing is permitted, There is a decision step of determining whether a suspicious device is involved. In the above control method, the display control stage includes, for example, (a) a secret image for displaying confidential information when it is determined that the suspicious device is not included in the one or more peripheral devices in the determination stage. , and an obstructing image for obstructing the viewing of at least a part of the concealed image, and a step of determining to execute a first concealing process for displaying on the display device in a time division manner. In the above control method, the display control step includes, for example, when it is determined in the (b) determination step that the one or more peripheral devices include a suspicious device, concealment is performed in a manner different from the first concealment processing. The step of determining to perform a second masking process to mask at least part of the information is included. In the above control method, the second anonymization process includes, for example, a process for stopping display of the confidential image on the display device, a process for changing the timing at which the confidential image is displayed on the display device, Processing for shrinking the displayed window, processing for moving the window behind other windows, processing for changing the display position of the window, and processing for changing the color tone or brightness of the hidden image including at least one of

In a fourth aspect of the invention, a program is provided. The program may be a program for causing a computer to function as the control device according to the first aspect. The program may be a program for causing a computer to execute the control method according to the third aspect. A computer readable medium storing the above program may be provided. The computer-readable medium may be non-transitory computer-readable medium. A computer-readable medium may be a computer-readable recording medium.

It should be noted that the above summary of the invention does not list all the necessary features of the invention. Subcombinations of these feature groups can also be inventions.

An example of the system configuration of the information concealment system 100 is shown schematically. An example desktop image 200 is shown schematically. An example output frame 302 is schematically shown. An example of the operation of display 122 and light shutter 142 is shown schematically. An example of a desktop image 500 that a third party 32 will perceive is schematically shown. An example output frame 602 is schematically shown. An example desktop image 700 as perceived by third party 32 is schematically shown. An example of the internal configuration of the display terminal 120 is shown schematically. An example of an internal configuration of an image display control unit 864 is shown schematically. An example of a data table 1000 is shown schematically. An example of the internal configuration of the output image generator 940 is shown schematically. An example of a superimposed state of the image 1200 is schematically shown. An example of correction processing by the image correction unit 1134 is schematically shown. An example of composition processing by the image composition unit 1140 is schematically shown. Another example of synthesizing processing by the image synthesizing unit 1140 is schematically shown. Another example of synthesizing processing by the image synthesizing unit 1140 is schematically shown. An example of an internal configuration of an image output management unit 950 is shown schematically. An example of information processing by the display timing determination unit 1732 is schematically shown. Another example of information processing by the display timing determination unit 1732 is schematically shown. An example of the internal configuration of the eyewear type device 140 is schematically shown. An example of the system configuration of the computer 3000 is shown schematically.

Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. Also, not all combinations of features described in the embodiments are essential for the solution of the invention. In addition, in the drawings, the same or similar parts may be denoted by the same reference numerals to omit redundant description.

[Overview of information concealment system 100]
FIG. 1 schematically shows an example of the system configuration of an information concealment system 100. As shown in FIG. In this embodiment, the information concealment system 100 is composed of a display terminal 120 and an eyewear device 140 . In this embodiment, the display terminal 120 comprises a display 122 . In this embodiment, eyewear-type device 140 includes optical shutter 142 .

In this embodiment, one or more third parties 32 and one or more third parties 34 exist around the display terminal 120 used by the user 20, and the display 122 of the display terminal 120 shows the user 20 Details of the information concealment system 100 will be described by taking as an example a case where information that the third parties 32 and 34 do not wish to view is displayed. Also, in this embodiment, the user 20 views the display 122 while wearing the eyewear device 140 . This places the light shutter 142 between the display 122 and the user 20 . On the other hand, the third person 32 looks into the display 122 without using the eyewear type device 140 or the eyewear type device 150 . Also, the third person 34 looks into the display 122 while wearing the eyewear type device 150 .

In this embodiment, for example, when the user 20 of the display terminal 120 operates the display terminal 120 to display information on the display 122 of the display terminal 120, the information concealment system 100 conceals at least part of the information. processing for encryption (sometimes referred to as anonymization processing) is executed. The information anonymization system 100 may execute anonymization processing according to an instruction from the user 20 or may execute anonymization processing according to the status of the display terminal 120 . For example, the information concealment system 100 executes concealment processing in response to detection of the third party 32 or 34 around the display terminal 120 . This prevents information leakage to the third party 32 or 34 even if the third party 32 or 34 exists around the display terminal 120 .

In one embodiment, the information anonymization system 100 performs a first anonymization process for suppressing information leakage to the third party 32 who is not wearing the eyewear type device 140 or the eyewear type device 150 as an anonymization process. Execute the process. In another embodiment, the information concealment system 100 includes a second system for suppressing not only information leakage to the third party 32 but also information leakage to the third party 34 wearing the eyewear type device 150 . Anonymization processing is executed. The second anonymization process may be a process of anonymizing at least part of the confidential information in a manner different from that of the first anonymization process.

As a method for preventing information leakage to a third party 32 and/or a third party 34 existing around the display terminal 120, the user 20 uses a wearable terminal equipped with a display device for displaying information. can be considered to be worn. A display, a projector, and the like are exemplified as the display device. Then, it is conceivable that the information concealment system 100 causes the display device of the wearable terminal to display information and stops the display of information on the display 122 of the display terminal 120 . However, a wearable terminal equipped with a display device is relatively heavy and imposes a heavy physical burden on the user 20 .

[First Anonymization Processing]
Therefore, according to this embodiment, the display terminal 120 executes the anonymization process according to the first embodiment according to the following procedure. For example, the display terminal 120 first displays an image (sometimes referred to as an original image) for displaying information (sometimes referred to as confidential information) to be subjected to anonymization processing for each frame. ) and an image (sometimes referred to as an obscuration image) for obstructing the viewing of at least a part of the original image.

The above-described anonymization processing may be performed in units of dots, in units of lines, in units of BOXes, in units of windows, or in units of screens. good. The above-described anonymization processing may be performed on a color-by-color basis.

Next, the display terminal 120 displays the original image and the obscuration image corresponding to each frame on the display 122 in a time-division manner during the period during which each frame is processed (sometimes referred to as a frame period). Let The number of times the original image corresponding to the frame is displayed on the display 122 during the frame period of a single frame may be one or more times. Similarly, during the frame period of a single frame, the occultation image corresponding to the frame may be displayed on display 122 once or multiple times. Even if the number of times the original image corresponding to the frame is displayed on the display 122 and the number of times the occlusion image corresponding to the frame is displayed on the display 122 during the frame period of a single frame are the same. Well, it can be different.

When the number of frames processed per unit time (sometimes referred to as frame rate) or the number of original images and obscuration images displayed per unit time exceeds a threshold, the switching interval between the original image and the obscuration image becomes shorter, and the third person 32 who is not wearing the eyewear device 140 or the eyewear device 150 cannot recognize the switching between the original image and the obscure image. In other words, as a stimulus sensed by the nerves of the third person 32, it is possible to distinguish between differences in brightness and/or color tone of light due to image switching. can't In this state, the third person 32 recognizes an image (sometimes referred to as a fused image) that is different from the original image, in which the original image and the obscuration image are fused by an afterimage.

For example, when the switching interval between the original image and the obscuration image is shorter than 1/10 second to 1/15 second, the display 122 appears to the third person 32 to display a fused image. As a result, it becomes difficult for the third party 32 to recognize the contents of the confidential information. Also, the shorter the switching interval between the original image and the obscuration image, the more difficult it becomes for the third party 32 to recognize the contents of the confidential information from the fusion image. For example, if the switching interval between the original image and the obscuration image is shorter than about 1/50 second, in many cases the third party 32 cannot recognize the content of the confidential information.

[Occultation image]
The obscuration image is an image that obstructs the viewing of at least part of the original image. is generated by The obscuration image is generated, for example, by converting the color of each pixel of the original image based on the correspondence relationship between the colors of the original image and the colors of the obscuration image. The above correspondence relationship may be represented by a function or by a table. When a color to be treated as a transparent color or transparent color (sometimes called a key color) is set, so that the color to be treated as a key color is not included in the colors used to generate the obscuration image. A correspondence between the colors of the original image and the colors of the obscuration image may be determined.

The obscuration image is not particularly limited as long as it is an image that can obstruct the visibility of confidential information included in the original image when the original image and the obscuration image are merged due to an afterimage. The obscuration image may be, for example, (i) an image of the original image that is displayed in substantially the entire image display area of the display 122, and (ii) an image that is displayed on the display 122 of the original image. An image for obstructing the viewing of the image displayed substantially in the center of the area, (iii) an image for obstructing the viewing of the image displayed on a specific scanning line of the display device among the original images, and ( iv) including at least one image for obstructing the viewing of an image having a particular color tone or brightness among the original images; The obscuration image includes (i) an image of the original image that is displayed in substantially the entirety of the image display area of the display 122, and (ii) an image of the original image that covers the image display area of the display 122. (iii) an image for obstructing the viewing of an image displayed substantially in the center, (iii) an image for obstructing the viewing of an image displayed on a specific scanning line of the display device among the original images, or (iv) It may be an image for obstructing the visual recognition of an image having a specific color tone or luminance among the original images.

The obscuration image may be a complementary color image (sometimes referred to as an inverted image) having a complementary color relationship with the original image. The complementary color relationship described above may be an example of the correspondence relationship between the colors of the original image and the colors of the obscuration image. The above complementary color relationship may be (i) a complementary color relationship on the color wheel (sometimes referred to as a first complementary color relationship), and (ii) the original image and the obscuration image to the viewer of the display 122. It may be a complementary color relationship (sometimes referred to as a second complementary color relationship) considering recognition of the fused image.

Complementary color relationships that take into account the visual characteristics of the viewer of the display 122 and the display characteristics of the display 122 are exemplified as complementary color relationships that allow the viewer of the display 122 to recognize the fusion image of the original image and the obscuration image. be. The display characteristics of the display 122 include the characteristics of light intensity adjustment processing of each pixel. The light intensity adjustment processing of each pixel includes (i) gamma correction processing or contrast suppression processing in cathode ray tube displays, and (ii) input/output in accordance with color space standards such as sRGB in liquid crystal displays, organic EL displays, and the like. Adjustment processing and the like are exemplified.

[First complementary color relationship]
For example, if the color of each pixel is represented by RGB colors, and one byte is assigned to each of RGB, according to the complementary color relationship on the color wheel, RGB=(255, 0, 0). The complementary color of red is light blue represented by RGB=(0, 255, 255). Similarly, according to the complementary color relationship on the color wheel, the complementary color of white represented by RGB=(255, 255, 255) is black represented by RGB=(0, 0, 0). If the color of the obscuration image corresponding to the color of the original image is determined based on the complementary color relationship on the color wheel, the generation procedure of the obscuration image is relatively simple, and the generation time of the obscuration image can be shortened.

[Second complementary color relationship]
On the other hand, according to the present embodiment, anonymization processing is performed by making the viewer of the display 122 recognize the fusion image of the original image and the obscuration image. Therefore, in the processing for determining the color of the obscuration image corresponding to the color of the original image, the accuracy of the concealment processing can be improved by considering the characteristics of the display 122 and the characteristics of human vision. For example, (i) the difference between the color indicated by the image data of the original image (for example, data in sRGB format) and the color displayed on the display 122, and/or (ii) the color displayed on the display 122. Anonymization processing is performed by determining the correspondence relationship between the colors of the original image and the colors of the obscuration image using the complementary color relationship defined in consideration of the difference between the colors recognized by the viewer and the colors recognized by the viewer. accuracy can be improved.

For example, the display 122 executes processing for adjusting the correspondence relationship between the input magnitude and the emission intensity of each color of RGB. In addition, human vision has a property (sometimes referred to as a visual characteristic) that it is sensitive to changes in low luminance and insensitive to changes in high luminance.

In general, in the display 122, when the parameter value of image data expressed in RGB format (sometimes referred to as an RGB value) is 0 or 255, the input and output correspond one-to-one. adjusted to Therefore, when the value of the parameter of the image data is 0 or 255, the emission intensity of the pixel of the display 122 corresponding to the parameter (sometimes referred to as the corresponding pixel) does not affect the color adjustment function of the display 122. I do not receive.

As described above, human vision has the property of being sensitive to changes in low luminance and insensitive to changes in high luminance. Therefore, in this case, the color obtained by averaging the RGB values of the colors of the original image and the obscuration image displayed on the display 122 and the color of the fusion image recognized by the viewer of the display 122 (when referred to as fusion color) ) does not match.

For example, if the color of the original image is white represented by RGB=(255, 255, 255) and the color of the obscuration image is black represented by RGB=(0, 0, 0), the display 122 The color obtained by averaging the RGB values of the colors of the original image and the obscuration image to be displayed is gray represented by RGB=(128, 128, 128). On the other hand, for example, when the gamma value of gamma correction by human vision is 0.45 and the gamma value of gamma correction in the display 122 is 2.2 (that is, 1/0.45), The color of the fusion image recognized by the viewer of the display 122 is gray represented by RGB=(186, 186, 186).

On the other hand, if one of the RGB values of a particular color is in the vicinity of 128, due to the influence of adjustment processing on the display 122 or the like, the viewer of the display 122 may see the particular color as a complementary color of the particular color. is difficult to determine with the naked eye. For example, if the above specific color is gray represented by RGB=(128, 128, 128), the display 122 adjusts the luminous intensity of the pixels in consideration of, for example, human visual characteristics. be. Thus, although the colors displayed on the display 122 are different from the gray represented by RGB=(128, 128, 128), the colors recognized by the viewer of the display 122 are RGB=(128, 128, 128). ) becomes gray.

Also, in this case, the color that is complementary to the specific color on the color wheel is gray represented by RGB=(127, 127, 127). Even in the case of gray represented by RGB=(127, 127, 127), the color recognized by the viewer of the display 122 is gray represented by RGB=(127, 127, 127). However, normally, a viewer of display 122 cannot distinguish between gray represented by RGB=(128, 128, 128) and gray represented by RGB=(127, 127, 127).

Considering the above, the method for determining the second complementary color relationship is as follows: (ii) at least one parameter of the RGB values of the fused color is determined to be the number of tones of that parameter; A conceivable method is to determine the complementary color of each of the one or more colors used to draw the image so as to be A times the median value (A is a positive number of 1.4 or more).

A value in the vicinity of the median value may be 98% or more and 102% or less of the median value. When the number of gradations is 256, the values of the above parameters may be 126-129, or may be 127 or 128. The above coefficient A may be 1.45 or more, or 1.5 or more. The coefficient A may be 1.4 or more and 1.6 or less, or 1.45 or more and 1.55 or less. When the number of gradations is 256, the values of the above parameters may be 183-189, 185-187, or 186.

When the complementary color of each color is determined so that at least one parameter of the RGB values of the fused color becomes the median value of the number of gradations of the parameter, in the concealment process, the obscuration image based on the second complementary color relationship In addition to the procedure for generating , a procedure for adjusting the colors of the original image is executed. In this case, the number of gray levels available for generating the obscuration image may be the same as the number of gray levels available for generating the original image. Thereby, confidential information can be concealed more reliably.

On the other hand, when at least one parameter among the RGB values of the fusion color determines the complementary color of each color so that it is 1.4 times or more the median value of the gradation number of the parameter, it can be used to generate an occultation image. The number of grayscales that can be used is less than the number of grayscales that can be used to generate the original image. However, in this case, the procedure for adjusting the color of the original image can be omitted in the anonymization process. This reduces the load on the computer for executing the anonymization process.

Other examples of the obscuration image include a monochrome image, a sandstorm image, a color bar image, and a display mode change image for changing the display mode of image elements included in the original image. A white image, a black image, a gray image, etc. are illustrated as a monochrome image. The display mode change image includes a position change image for changing the position (referred to as the display position) of the image element included in the original image on the display 122, and a position change image for changing the size of the image element included in the original image. , a shape-changed image for changing the shape of an image element included in the original image, and a color-changed image for changing the color tone and/or brightness of the image element included in the original image. . Examples of image elements include symbols, graphics, tables, pictures, photographs, and the like. Symbols are exemplified by letters, codes, numerals, marks, and the like.

A plurality of original images and occultation images may be used to generate an animation using image elements included in the original images. Examples of the above animation include an animation in which at least one of the one or more image elements included in the original image rotates, an animation in which at least one of the one or more image elements included in the original image repeats scaling, an original image An animation in which at least one of one or more image elements included in is blinking is exemplified. In an animation in which an image element included in an original image rotates, the center of rotation of the image element may be a point inside the image element or a point outside the image element.

The obscuration image may be an image for generating a second image element different from the first image element contained in the original image when the original image and the obscuration image are fused by an afterimage. The first image element included in the original image and the second image element included in the fusion image differ, for example, in at least one of color, pattern, and shape. The first image element included in the original image and the second image element included in the fusion image differ, for example, in at least one of position and size in each image.

For example, when the original image and the complementary color image are displayed on the display 122 in a short period of time in a time-division manner, the third person 32 sees a gray image in which both images are merged due to an afterimage. appear. This interferes with viewing of the original image. Note that the relationship between the color tone of each image of the complementary color image and the color tone of each image of the original image may or may not be a completely complementary color relationship. For example, by adjusting the color tone of the complementary color image, the color tone of the gray image can be adjusted.

For example, when the original image and the position-changed image are displayed on the display 122 in a short period of time in a time division manner, the original image appears blurred to the third party 32 due to the afterimage. This interferes with viewing of the original image.

On the other hand, according to this embodiment, the user 20 wears an eyewear type device 140 having an optical shutter 142 . The optical shutter 142 can transmit or block light by varying the transmittance of light. Examples of the optical shutter 142 include a liquid crystal shutter, a rotating disk type slide, and a reflection mirror by MEMS.

The eyewear device 140 controls the operation of the optical shutter 142 to perform processing for making the concealed confidential information recognizable (sometimes referred to as recognizable processing). Specifically, the eyewear device 140 controls the timing at which the original image and the obscuration image are displayed on the display 122 (sometimes referred to as display timing) and the timing at which the optical shutter 142 opens and closes (opening/closing timing). (sometimes referred to as ).

As a result, the optical shutter 142 opens at the timing when the original image is displayed on the display 122 . Also, the optical shutter 142 is closed at the timing when the obscuration image is displayed on the display 122 . In one embodiment, the opening and closing timing of the optical shutter 142 is adjusted so that the length of the period during which the original image is displayed on the display 122 and the length of the period during which the optical shutter 142 is open are the same. In another embodiment, the opening/closing timing of the optical shutter 142 is adjusted so that the optical shutter 142 is open during a part of the period during which the original image is displayed on the display 122 .

According to this embodiment, the obscuration image is blocked by the optical shutter 142 and the light that constitutes the obscuration image does not enter the eyes of the user 20 . This allows the user 20 to selectively view the original image. As a result, the user 20 can recognize the content of confidential information. Moreover, the optical shutter 142 has a simpler structure and consumes less power than a display device that displays an image. Therefore, the eyewear device 140 with the optical shutter 142 is much lighter than a wearable terminal with a display device. Thereby, according to this embodiment, information leakage to a third party can be suppressed while reducing the burden on the user 20 .

[Second Anonymization Processing]
Also, according to this embodiment, the display terminal 120 executes the anonymization process according to the second embodiment according to the following procedure. For example, the display terminal 120 first determines whether there is an eyewear device 150 around the display terminal 120 . The details of the method of determining whether or not the eyewear device 150 is present will be described later.

(a) When it is determined that the eyewear device 150 does not exist around the display terminal 120, the display terminal 120 determines to execute the above-described first anonymization process. The display terminal 120 is configured to be able to execute the first anonymization process described above. When the display terminal 120 executes the first anonymization process, information leakage to the third party 32 who is not wearing the eyewear device 140 or the eyewear device 150 is prevented.

Note that when it is determined that the eyewear device 150 does not exist around the display terminal 120 and that the third person 32 exists around the display terminal 120, the display terminal 120 performs the above-described You may perform the 1st anonymization process which was carried out. If it is determined that the eyewear device 150 and the third party 32 are not present around the display terminal 120, the display terminal 120 does not have to perform the above-described first anonymization processing.

On the other hand, (b) when it is determined that the eyewear device 150 exists around the display terminal 120, the display terminal 120 determines to execute the above-described second anonymization process. The display terminal 120 is configured to be able to execute the second anonymization process described above. When the display terminal 120 executes the second anonymization process, not only is information leakage to the third party 32 prevented, but information leakage to the third party 34 can also be suppressed.

The second anonymization processing includes (i) processing for stopping the display of the original image on the display 122, (ii) processing for changing the timing at which the original image is displayed on the display 122, and (iii) (iv) processing for moving the window displaying the confidential information behind other windows; (v) display position of the window displaying the confidential information; and (vi) changing the tone, contrast, or brightness of the original image. The processing for changing the timing at which the original image is displayed on the display 122 includes (i) changing a pattern (sometimes referred to as a display pattern) for defining display and non-display of the original image; (ii) processing for changing the start time of the pattern; and (iii) processing for changing the period of the pattern.

For example, when the display pattern defines that one original image is displayed for 3 ms and one obscuration image is displayed for 3 ms in each frame, the eyewear device 150 is placed around the display terminal 120. When it is determined that they exist, the display terminal 120 changes the display pattern and decides to display one original image for 3ms and one occultation image for 1ms for each frame. Next, display terminal 120 transmits information indicating the changed display pattern to eyewear device 140 in a manner that eyewear device 150 cannot receive.

The display terminal 120 may encrypt the information indicating the changed display pattern and transmit it to the eyewear device 140 . For example, the display terminal 120 encrypts information indicating the changed display pattern with a common key and transmits the encrypted information to the eyewear device 140 .

Thereby, the eyewear device 140 can block transmission of the obscuration image by the optical shutter 142 . Therefore, the user 20 wearing the eyewear device 140 can visually recognize the confidential information displayed in the original image. On the other hand, the optical shutter 152 of the eyewear-type device 150 cannot block transmission of all occult images. As a result, it becomes more difficult for the third person 34 wearing the eyewear-type device 150 to visually recognize the confidential information compared to the user 20 wearing the eyewear-type device 140 . The same applies when the start time of the display pattern is changed and when the period of the display pattern is changed.

[Determination process of anonymized range]
As mentioned above, the obscuration image is used to obscure the viewing of at least part of the original image. The area in the original image that should be obstructed from being viewed by a third party (that is, the area subject to anonymization processing and may be referred to as a concealment area) is the entire or substantially the entire original image. It may be an area where the change in color tone meets a predetermined condition, or it may be at least one of one or more windows displayed in the original image. The hidden area may be a specific area inside the window.

In this embodiment, the display terminal 120 first determines the conditions for determining the hidden area based on settings or instructions from the user. As described above, the conditions for determining the hidden region are: (i) the condition that the hidden region is the entire or substantially the entire original image; (iii) the hidden area is a display object (for example, an object drawn on the screen such as a text box), and the type and/or name of the display object; is a display object that satisfies a predetermined condition, and (iv) a condition that the hidden area is a window that satisfies the predetermined condition. The area where the color tone change meets the predetermined condition may be the area surrounded by the detected edges. Next, the display terminal 120 determines the hidden area of the original image based on the conditions for determining the hidden area.

[Anonymization processing of hidden area]
In one embodiment of the present technology, the display terminal 120 determines the range (for example, the range of the concealment area) in which the above-described concealment processing is performed, based on the change in color tone of the original image. For example, the display terminal 120 first identifies the position of the confidential information in the original image based on the degree of change in color tone of a plurality of pixels forming the original image. The display terminal 120 combines and evaluates known filtering processes (for example, convolution, feature amount extraction technology, etc.) for detecting symbols, figures, tables, pictures, photographs, etc., to obtain confidential information in the original image. can be located. The display terminal 120 may specify a region in which confidential information is not arranged in the original image by executing known filtering processing for detecting symbols, figures, tables, pictures, photographs, and the like.

For example, the display terminal 120 traces the entire area of the original image using a first-order differential filter, and determines a filter block with a large degree of edge as a hidden area. A known filter such as a prwitt filter may be employed as the first-order differential filter. By using the first-order differential filter, the entire area of the original image can be traced at high speed.

For example, the display terminal 120 uses, as training data, images such as characters, WORDs, short sentences, symbols, figures, tables, pictures, photographs, etc., or parts thereof, and small image parts obtained by cutting out parts from the images, to create text, An estimation model for estimating a hidden area is generated by machine learning from feature values of information that can be confidential information, such as charts and photographs. The display terminal 120 inputs the data of the original image to the estimation model, estimates even an image match that is not large in terms of probability as an interesting area, and uses the degree of density to detect a part of the original image as the position of the confidential information. and determine the hidden area. The display terminal 120 utilizes a trained convolutional neural network (CNN Convolutional Neural Network) to create a feature map using CNN in the world in which the original image is convoluted. may be

The display terminal 120 may determine an input item column arranged on the WEB page or an area including the input item column as a hidden area. Examples of input item fields include a field for inputting personal information such as name and date of birth, and a field for inputting login information such as login ID and password.

In this case, the display terminal 120 may identify the position or area where the input item column is displayed by analyzing the source of the web page. For example, the display terminal 120 analyzes the source of the WEB page and searches for an object whose name setting matches a predetermined condition. As a condition regarding the name setting, the condition that the object has a name setting such as "pass" and "DOB" in which confidential information can be entered is exemplified. When an object whose name setting matches a predetermined condition is extracted, the display terminal 120 determines the display position of the object as a hidden area, for example.

The display terminal 120 may specify the display position of the name of the object in the original image by analyzing the original image and specifying the position of one or more character strings displayed in the original image. . The display terminal 120 may determine the hidden area by specifying the position or area where the input item field is displayed based on the display position of the name of the object in the original image. The display terminal 120 determines, for example, an area of a predetermined shape and size including the display position of the name of the object in the original image as a hidden area. The display terminal 120 may determine the position, size and range of the hidden area so that the hidden area covers the input item field.

The display terminal 120 may determine the hidden area based on the time-series variation of a plurality of original images. For example, the display terminal 120 analyzes a plurality of original images displayed at different timings. When a change is detected in at least part of a plurality of original images, display terminal 120 may determine an area with a degree of time-series variation greater than a predetermined reference as an anonymization area.

Display terminal 120 determines a plurality of candidate areas that are candidates for anonymized areas by executing the above-described process of determining a plurality of anonymized areas. In one embodiment, the display terminal 120 may determine the anonymization area based on how multiple candidate areas are superimposed. For example, display terminal 120 determines an area where n (n is a positive integer equal to or greater than 2) candidate areas overlap as an anonymization area. In another embodiment, the display terminal 120 may determine an area formed by ORing multiple candidate areas as an anonymization area.

Next, display terminal 120 determines the range of each of the one or more hidden areas such that at least part of the confidential information is covered by the one or more hidden areas. Examples of the range of the hidden area include the position of the hidden area and the shape and size of the hidden area. For example, when the confidential information is a character string of multiple lines, the display terminal 120 determines a plurality of rectangular areas corresponding to each line of the multiple character strings as the confidential area. If the confidential information is a character string of multiple lines, a single rectangular area corresponding to the multiple character strings may be determined as the confidential area.

Also, when the range of the hidden area in the original image is determined, the display terminal 120 generates an obscuration image for obstructing the visibility of the hidden area. As described above, the display terminal 120 generates an occultation image by converting the color tone of at least part of the one or more pixels arranged inside the hidden area of the original image.

In one embodiment, display terminal 120 may generate an occultation image without converting the tone of one or more pixels located outside the hidden region of the original image. This improves the speed of the obscuration image generation process.

In another embodiment, the display terminal 120 may generate an occultation image by adjusting the brightness or brightness of at least a portion of one or more pixels arranged outside the hidden area of the original image. For example, when the original image and the obscure image are displayed in a time division manner, the inside of the hidden area may feel darker than the outside of the hidden area depending on the user 20 due to human visual characteristics, display characteristics of the display 122, and the like. Sometimes. Therefore, the display terminal 120 may generate an obscuration image by reducing the brightness or brightness of at least some of the one or more pixels arranged outside the concealment area of the original image, compared to the original image. Note that the display terminal 120 may generate an obscuration image by increasing the brightness or luminance of at least some of the one or more pixels arranged inside the concealment area of the original image, compared to the original image. Thereby, the discomfort felt by the user 20 is reduced.

After that, the display terminal 120 controls the display 122 so that the original image and the obscuration image are displayed on the display 122 in a time division manner. According to this embodiment, part of the original screen is subject to anonymization processing. As a result, compared to the case where the entire original screen is subject to the anonymization process, the area of the area to be the anonymization process target becomes smaller. As a result, flickering perceived by the user 20 when the original image and the obscuration image are displayed on the display 122 in a time division manner is reduced.

[Anonymization processing for each window]
In another embodiment of the present technology, the display terminal 120 determines whether or not the above-described anonymization process is necessary for each window. For example, the display terminal 120 determines a window displaying confidential information (sometimes referred to as a target window) among the one or more windows arranged on the desktop screen as a target for anonymization processing. The display terminal 120 determines, for example, the target window or the image display area of the target window as the hidden area. Display terminal 120 may determine the target window or an area covering the image display area of the target window as the hidden area.

As described above, display terminal 120 may determine a specific area inside the target window as a target for anonymization processing. For example, the display terminal 120 determines the hidden area inside the target window based on the change in the color tone of the image in the image display area of the target window. For example, the display terminal 120 first identifies the position of the confidential information inside the target window based on the degree of change in color tone of a plurality of pixels forming the original image. The display terminal 120 may specify the position of the confidential information inside the target window by executing a known filtering process for detecting symbols, graphics, tables, pictures, photographs, and the like. Display terminal 120 may specify an area in which confidential information is not arranged within the target window by executing a known filtering process for detecting symbols, graphics, tables, pictures, photographs, and the like. Next, display terminal 120 determines the range of each of the one or more hidden areas such that at least part of the confidential information is covered by the one or more hidden areas. Examples of the range of the hidden area include the position of the hidden area and the shape and size of the hidden area.

Also, when the range of the hidden area in the original image is determined, the display terminal 120 generates an obscuration image for obstructing the visibility of the hidden area. As described above, the display terminal 120 generates an occultation image by converting the color tone of at least part of the one or more pixels arranged inside the hidden area of the original image. In one embodiment, display terminal 120 may generate an occultation image without converting the tone of one or more pixels located outside the hidden region of the original image. In another embodiment, the display terminal 120 may generate an occultation image by adjusting the brightness or brightness of at least a portion of one or more pixels arranged outside the hidden area of the original image.

The display terminal 120 may control the display 122 so that the original image and the obscuration image are displayed in a time division manner in a region corresponding to the target window on the display 122 . Thereby, anonymization processing can be executed in units of windows. According to the present embodiment, since the anonymization process is executed on a window-by-window basis, the area of the area to be the anonymization process target becomes smaller than when the entire screen is an anonymization process target. This reduces flickering perceived by the user 20 when the original image and the obscuration image are displayed on the display 122 in a time division manner.

According to this embodiment, the user 20 performs an input operation for specifying a window for which anonymization processing is desired, or selects a window for displaying the execution result of a specific application program as an object for anonymization processing. By executing the setting operation for specifying, it is possible to easily specify the area where the original image and the obscuration image are displayed on the desktop screen. Also, even when the user 20 changes the position of the target window (sometimes referred to as moving the target window) or changes the shape and/or size of the target window, the display terminal 120 can adjust the areas where the original image and the obscure image are displayed on the desktop screen according to the change. As a result, even if the user 20 changes the position, shape and/or size of the target window, the user 20 can re-specify the area where the original image and the obscuration image are displayed on the desktop screen. This prevents confidential information from being leaked.

Also, for example, when (a) user 20 and third party 32 and/or third party 34 view display 122 together, or (b) user 20 views third party 32 and/or third party When sharing a screen with a third party 34, on the desktop screen of the display terminal 120 used by the user 20, a window that can be disclosed to the third party 32 and/or the third party 34, and the third party 32 and/or Windows that cannot be disclosed to the third party 34 may coexist. Even in such a case, according to the present embodiment, the user 20 does not execute the anonymization processing for the windows that can be disclosed, but executes the anonymization processing for the windows that cannot be disclosed. can be done.

[Detection processing of eyewear type device 150]
In another embodiment of the present technology, the display terminal 120 determines whether a device having at least one of an optical shutter and an imaging device exists around the display terminal 120 . For example, the display terminal 120 executes detection processing of the eyewear type device 140 and the eyewear type device 150 existing around the display terminal 120 .

When a device having at least one of an optical shutter and an image pickup device is detected around the display terminal 120, the display terminal 120 determines whether or not the device is permitted to be recognized. . If the above device is not a device for which recognizability processing is permitted, display terminal 120 executes the above-described second anonymization processing. As a result, even if there is a third party wearing the eyewear device 150 around the display terminal 120, leakage of confidential information can be prevented.

As described above, when the second anonymization process is executed, the display of the confidential information on the display 122 is stopped, or the timing at which the confidential information is displayed on the display 122 (sometimes referred to as display timing). ) is changed. The display terminal 120 may determine the details of the process to be executed according to the level of risk of information leakage. The risk level includes (i) the type of application generating confidential information, (ii) the number of detected eyewear devices 150, and (iii) the display terminal 120 and the detected eyewear devices 150. , and the like.

The display terminal 120 may output a warning to the user 20 if the above device is not a device for which the recognizability process is permitted. For example, the display terminal 120 outputs a warning sound or warning message from the audio output device of the display terminal 120 . For example, display terminal 120 displays a warning icon or warning message on display 122 . The display terminal 120 may turn on a warning light arranged on the display terminal 120 . As a result, even if there is a third party wearing the eyewear device 150 around the display terminal 120, leakage of confidential information can be prevented.

In addition, the display terminal 120 may temporarily hide windows, folders, or icons related to confidential information based on instructions from the user 20 who received the warning. Examples of processing to temporarily hide a window, folder, or icon include processing to hide the window, processing to minimize the window, processing to move the window behind other windows, etc. be done. As a result, even if there is a third party wearing the eyewear device 150 around the display terminal 120, leakage of confidential information can be prevented.

For example, the display terminal 120 notifies the eyewear device 140 of information indicating the changed display timing by transmitting a control signal. The eyewear device 140 adjusts the opening/closing timing of the optical shutter 142 based on the information indicating the changed display timing. Thereby, the user 20 wearing the eyewear type device 140 can continue to view the confidential information displayed on the display 122 . On the other hand, the eyewear-type device 150 cannot adjust the opening/closing timing of the optical shutter 152 in accordance with the changed display timing, or the adjustment requires time. This prevents confidential information from being leaked.

According to this embodiment, execution of the second anonymization process is stopped when a predetermined event is detected after the second anonymization process is executed. As a result, for example, the display terminal 120 returns to the state in which the first anonymization process is being performed. Predetermined events include the elapse of a predetermined period of time and the non-detection of a device having at least one of an optical shutter and an imaging device around the display terminal 120 .

[Overview of Each Unit Related to Information Concealment System 100]
In this embodiment, the display terminal 120 causes the display 122 to display various information. The display terminal 120 also transmits and receives various control signals to and from the eyewear device 140 . Examples of the display terminal 120 include a personal computer and a mobile terminal. Examples of mobile terminals include mobile phones, smart phones, PDAs, tablets, notebook or laptop computers, wearable computers, and the like. As a wearable computer, a bracelet-type or ring-type wearable computer is exemplified.

In this embodiment, the eyewear device 140 is a device that is permitted by the user 20 or the information concealment system 100 to process information displayed on the display 122 of the display terminal 120 so that it can be recognized. The eyewear device 140 controls the opening/closing timing of the optical shutter 142 . The eyewear device 140 may be configured to be able to adjust the opening/closing timing of the optical shutter 142 . Also, the eyewear device 140 transmits and receives various control signals to and from the display terminal 120 . The eyewear type device 140 may be a spectacle type device, a head mounted type device, or a contact lens type device.

In this embodiment, the eyewear device 140 is a device that is not permitted by the user 20 or the information concealment system 100 to process information displayed on the display 122 of the display terminal 120 so that it can be recognized. The eyewear device 150 controls the opening/closing timing of the optical shutter 152 . The eyewear device 150 may be configured so that the opening/closing timing of the optical shutter 142 can be adjusted. The eyewear type device 150 may be a spectacle type device, a head mounted type device, or a contact lens type device.

[Specific configuration of each part of information concealment system 100]
Each part of the confidential information system 100 may be realized by hardware, software, or both hardware and software. At least a part of each part of the confidential information system 100 may be implemented by a single server, or may be implemented by a plurality of servers. At least part of each unit of the information concealment system 100 may be realized on a virtual machine or a cloud system. At least a part of each part of the confidential information system 100 may be realized by a personal computer or a mobile terminal. Examples of mobile terminals include mobile phones, smart phones, PDAs, tablets, notebook computers or laptop computers, wearable computers, and the like. Each part of the information concealment system 100 may store information using a distributed ledger technology such as blockchain or a distributed network.

When at least some of the components that make up the confidential information system 100 are implemented by software, the components implemented by the software define operations related to the components in an information processing apparatus with a general configuration. It may be implemented by activating the software or program that runs the The information processing apparatus having the above general configuration includes (i) a data processing apparatus having a processor such as a CPU and GPU, a ROM, a RAM, a communication interface, and the like; and (ii) a keyboard, pointing device, touch panel, camera, voice input Input devices such as devices, gesture input devices, various sensors, and GPS receivers, (iii) output devices such as display devices, audio output devices, and vibration devices, and (iv) storage devices such as memories, HDDs, and SSDs (external including a storage device).

In the information processing apparatus having the general configuration described above, the data processing apparatus or storage device may store the software or program described above. The software or program is executed by the processor to cause the information processing device to perform operations defined by the software or program. The above software or program may be stored in a non-transitory computer-readable recording medium. The above software or program may be a program for causing a computer to function as information concealment system 100 or a part thereof. The above software or program may be a program for causing a computer to execute the information processing method in the information concealment system 100 or part thereof.

In one embodiment, the information processing method described above may be a control method for controlling a display device having a display device for displaying images. The above information processing method may be a control method for a computer to control the above display device.

The control method described above has, for example, a secret image generation step of generating a secret image for displaying secret information, which is information to be subjected to anonymization processing. The above control method has, for example, an obstruction image generation step of generating an obstruction image for obstructing viewing of at least part of the confidential image. The control method described above includes, for example, a display control step of controlling the display device so that the display device displays the confidential image and the disturbing image in each frame in a time division manner.

In another embodiment, the information processing method described above may be a control method for controlling a display device having a display device for displaying images. The above information processing method may be a control method for a computer to control the above display device.

The control method described above comprises, for example, a display control stage for controlling the display device. The above control method has, for example, at least one of an optical shutter and an imaging device, and has a peripheral device detection step of detecting one or more peripheral devices existing around the display device. In the above control method, for example, among the one or more peripheral devices detected in the peripheral device detection stage, a device that does not meet the genuine product determination condition, which is a condition regarding the device for which confidential information recognition processing is permitted, There is a decision step of determining whether a suspicious device is involved.

In the above control method, the display control stage includes, for example, (a) a secret image for displaying confidential information when it is determined that the suspicious device is not included in the one or more peripheral devices in the determination stage. , and an obstructing image for obstructing the viewing of at least a part of the concealed image, and a step of determining to execute a first concealing process for displaying on the display device in a time division manner. In the display control stage, for example, when it is determined in the (b) determination stage that one or more peripheral devices include a suspicious device, at least part of the confidential information is displayed in a manner different from the first anonymization process. The step of deciding to execute a second anonymizing process for anonymizing is included. The second anonymization process includes, for example, a process for stopping display of the confidential image on the display device, a process for changing the timing at which the confidential image is displayed on the display device, and a window in which the confidential information is displayed is reduced. , a process for moving a window to the back of another window, a process for changing the display position of the window, and a process for changing the color tone or brightness of the hidden image.

In still another embodiment, the above information processing method is a control method for controlling a display device having a display device for displaying an image including one or more display windows for displaying information presented to a user. can be The above information processing method may be a control method for a computer to control the above display device.

The control method described above has, for example, a secret image generation step of generating a secret image for displaying secret information, which is information to be subjected to anonymization processing. The control method described above has, for example, an obstruction image generating step of generating an obstruction image for obstructing viewing of the confidential image. The control method described above includes, for example, a display control step of controlling the display device so that the display device displays the confidential image and the disturbing image in each frame in a time division manner. In the above control method, the confidential image includes at least an image of the image display area of the display window for displaying confidential information. The above control method may include the step of determining whether or not the anonymization process is necessary in units of windows.

The display terminal 120 may be an example of a display device. Display 122 may be an example of a display device. Eyewear-type device 140 may be an example of a peripheral device, an authorized device, or a device that is permitted to perform confidential information recognizable processing. Eyewear-type device 150 may be an example of a peripheral or suspicious device.

Information that the user 20 does not want the third parties 32 and 34 to view may be an example of confidential information. The original image may be an example of the display image. An occulting image may be an example of an obstructing image. The first anonymization process may be an example of the first anonymization process. The second anonymization process may be an example of the second anonymization process. The anonymization process according to the first embodiment may be an example of the first anonymization process. The anonymization process according to the second embodiment may be an example of the second anonymization process. The visual characteristics of a viewer of display 122 may be an example of human visual characteristics. The area where the confidential information is displayed in the original image may be an example of at least a part of the confidential image.

[An example of another embodiment]
In the present embodiment, an example of the information concealment system 100 has been described with the display terminal 120 having the display 122 as an example. However, the information concealment system 100 is not limited to this embodiment. In other embodiments, display terminal 120 may comprise various devices capable of displaying images. For example, display terminal 120 includes a projector that displays an image by projecting the image.

In this embodiment, the user 20 and the third party 32 and/or the third party 34 view the same display 122, and the display terminal 120 used by the user 20 displays the original image displayed on the display 122. An example of the information concealment system 100 has been described, taking the case of generating an occultation image as an example. However, the information concealment system 100 is not limited to this embodiment.

In other embodiments, user 20 and third party 32 and/or third party 34 utilize different display terminals 120, and display terminal 120 utilized by user 20 and third party 32 and/or third party The display terminal 120 used by the third party 34 and the information processing device that provides the image data generate the original image and the obscuration image. As the above information processing apparatus, a server configured to be able to communicate with these display terminals 120 is exemplified. In this case, the information processing apparatus may transmit a control signal for controlling the operation of the eyewear device 140 to the display terminal 120 used by the user 20. / Alternatively, the control signal may not be transmitted to the display terminal 120 used by the third party 34 .

In this embodiment, as an example, a third party 34 uses an eyewear device 150 equipped with an optical shutter 152 to peek at confidential information displayed on the display 122. An example was described. However, the mode of peeking at confidential information is not limited to this embodiment.

In another embodiment, the third party 34 peeks at the confidential information displayed on the display 122 using a device having an imaging device that captures an image of the display 122 on which the confidential information is displayed. Even in this case, if it is determined that the device provided with the image pickup device is not a device for which the above-described recognition enablement process is permitted, the display terminal 120 displays the confidential information on the display 122. to stop. This suppresses information leakage.

In the present embodiment, an example of the information concealment system 100 has been described by taking as an example the case where the eyewear type device 140 has the optical shutter 142 and the obscuration image is blocked by the optical shutter 142 . However, the information concealment system 100 is not limited to this embodiment. In another embodiment, the information concealment system 100 or the eyewear device 140 (i) uses an imaging device such as an image sensor to capture an image of the display 122 that displays the concealment image and the concealment image, and (ii) the captured image. and (iii) presenting the extracted confidential image to the user.

FIG. 2 schematically shows an example desktop image 200 for displaying information. Desktop image 200 may be an example of an image displayed on display 122 when anonymization processing is not being performed. Desktop image 200 is displayed, for example, on the screen display area of display 122 .

In this embodiment, the desktop image 200 has a desktop display area 212 and a taskbar display area 214 . In this embodiment, desktop display area 212 displays an image of active window 220 , an image of inactive window 240 , and an image of pointer 260 . In this embodiment, active window 220 has a title area 222 and a work area 224 . A cursor 226 is placed at the current input position in the work area 224 . Similarly, inactive window 240 has a title area 242 and a work area 244 .

In this embodiment, one or more windows are displayed in the desktop display area 212 . Each of the one or more windows displayed in desktop display area 212 may be a display area assigned to an application executed on display terminal 120, and may be a display area configured to communicate with display terminal 120. It may be a display area allocated for an application running on a computer.

On the other hand, in the present embodiment, the taskbar display area 214 may be an area in which the window image is not displayed. The taskbar display area 214 displays, for example, the icon of the application currently being executed on the display terminal 120, the time, and the like.

In this embodiment, active window 220 displays information presented to user 20 by the corresponding application program. The active window 220 may be the window currently being operated by the user 20 among the one or more windows displayed in the desktop display area 212 . In this embodiment, the title area 222 displays the title name of the active window 220 . In addition, the content of the current work is displayed in the work area 224 . Cursor 226 indicates the current input position in work area 224, for example.

In this embodiment, inactive window 240 displays information presented to user 20 by the corresponding application program. The inactive window 240 may be one or more windows displayed in the desktop display area 212 that are not currently being operated by the user 20 . In this embodiment, the title area 242 displays the title name of the inactive window 240 . In the work area 244, the content of the last work performed by the user 20 on the inactive window 240 is displayed.

In this embodiment, the pointer 260 indicates the position indicated by the pointing device. Examples of pointing devices include a mouse and a touch panel.

Desktop image 200 may be an example of a confidential image. Active window 220 may be an example of a display window. Work area 224 of active window 220 may be an example of an image display area of a display window. The image of active window 220 may be an example of a confidential image. The image in working area 224 of active window 220 may be an example of a hidden image. Inactive window 240 may be an example of a display window. Work area 244 of inactive window 240 may be an example of an image display area of a display window. The image of the inactive window 240 may be an example of a hidden image. The image of work area 244 of inactive window 240 may be an example of the image display area of the display window.

An example of anonymization processing in the information anonymization system 100 will be described with reference to FIGS. 3, 4, and 5. FIG. According to the present embodiment, an example of anonymization processing will be described by taking as an example a case where information displayed in the active window 220 in the desktop image 200 described with reference to FIG. 2 is anonymized.

FIG. 3 schematically illustrates an example output frame 302 that stores data for an image displayed on display 122 . FIG. 4 schematically shows an example of operations of the display 122 and the optical shutter 142 during execution of anonymization processing. FIG. 5 schematically shows an example desktop image 500 as perceived by third party 32 .

As shown in FIG. 3, in this embodiment, output frame 302 includes subframe 312 and subframe 314 . Display terminal 120 processes subframe 312 to display desktop image 320 in desktop display area 212 of display 122 . Similarly, display terminal 120 processes sub-frame 314 to cause desktop image 340 to be displayed in desktop display area 212 of display 122 .

In this embodiment, desktop image 320 includes window image 322 , window image 324 , and pointer image 326 . Window image 322 may be the image of active window 220 at the time output frame 302 is processed. Window image 324 may be an image of inactive window 240 at the time described above. Pointer image 326 may be an image of pointer 260 at that point in time.

In this embodiment, desktop image 340 includes window image 342 , window image 344 , and pointer image 346 . Window image 342 may be the image of active window 220 at the time output frame 302 is processed. Window image 344 may be an image of inactive window 240 at the time described above. Pointer image 346 may be an image of pointer 260 at that point in time.

In this embodiment, the desktop image 320 is the original image of the output frame 302, and the desktop image 340 is the obscuration image of the output frame 302. An example of anonymization processing in the information anonymization system 100 will be described. . Desktop image 340 may be the same image as desktop image 320 , except that window image 342 is displayed instead of window image 322 . A window image 322 is the original image of the active window 220 , and a window image 342 is an obscuration image of the window image 322 .

For example, if the output frame 302 is one frame of a moving image reproduced in the active window 220, the window image 322 displays the original image of the one frame image. On the other hand, in the window image 342, an obscuration image of the one-frame image is displayed. As described above, according to the present embodiment, for each frame of a moving image, the original image and the obscuration image related to the same frame are displayed in a time-division manner so as to form a pair. Therefore, the third person 32 not wearing the eyewear type device 140 recognizes, for example, a gray monochromatic image as the fusion image. As a result, the content of the moving image reproduced in the active window 220 is made anonymous.

In this embodiment, window image 322 is an image for displaying information displayed in active window 220 and includes information presented to user 20 by an application program corresponding to active window 220 . The window image 322 is, for example, an input image generated based on the input image signal output by the above application program.

In this embodiment, the window image 342 is an image for obstructing the visual recognition of the window image 322, and is, for example, a complementary color image having a complementary color relationship with the window image 322. FIG. When the input image and the complementary color image are alternately switched at high speed, a third party 32 not wearing the eyewear-type device 140 or the eyewear-type device 150 will see, on display 122, due to the persistence of the input image and the complementary color image, A gray image in which both images are fused appears to be displayed.

As shown in FIG. 4, according to the present embodiment, display terminal 120 processes a plurality of output frames having a configuration similar to output frame 302 so that display 122 displays desktop image 320 and desktop image 340. Display in time division. For example, the display 122 displays the desktop image 320 during the period Pso during which the sub-frame 312 is processed in the period Pf during which the output frame 302 is processed (sometimes referred to as a frame period). Display 122 also displays desktop image 340 during period Psj when sub-frame 314 is processed.

According to this embodiment, in the eyewear device 140, the length of the open/close cycle Pg of the optical shutter 142 is equal to the length of Pf, and the length of the period Pgo during which the optical shutter 142 is open is equal to the length of Pso. Thus, the opening/closing timing of the optical shutter 142 is set so that the length of the period Pgc during which the optical shutter 142 is closed is equal to the length of Psj. The eyewear device 140 also controls the opening and closing of the optical shutter 142 according to the synchronization signal from the display terminal 120 . Thereby, the eyewear type device 140 opens the optical shutter 142 during the period Pso during which the subframe 312 is processed, and closes the optical shutter 142 during the period Pso during which the subframe 312 is processed.

As shown in FIG. 5 , a region of desktop image 500 corresponding to active window 220 of desktop image 200 appears to third person 32 to be obscured by fusion image 524 . If the window image 342 is a complementary color image of the window image 322, the fusion image 524 will be a gray image. On the other hand, the user 20 wearing the eyewear device 140 can visually recognize the desktop image 200 .

Desktop image 320 may be an example of a hidden image. Window image 322 may be an example of a confidential image. Information displayed in the window image 322 may be an example of confidential information. Desktop image 340 may be an example of a disturbing image. Window image 342 may be an example of an obstruction image. A complementary color image may be an example of an interfering image.

[An example of another embodiment]
In the present embodiment, (i) the information displayed in the active window 220 includes confidential information, and (ii) the window image 322 including the title area 222 and the work area 224 of the active window 220 is a confidential image, (iii) An example of the information concealment system 100 has been described by taking as an example the case where the window image 342, which has the same shape and size as the window image 322 and is a complementary color image of the window image 322, is an obstructing image. However, the information concealment system 100 is not limited to this embodiment.

In other embodiments, the covert image includes at least an image of working area 224 of active window 220 . For example, the information concealment system 100 executes concealment processing using the image in the work area 224 of the active window 220 as the concealment image. In this case, the information concealment system 100 generates, for example, a complementary color image of the image of the work area 224 of the active window 220 as the disturbing image. The information concealment system 100 may generate a complementary color image of an image of an area larger than the working area 224 including the working area 224 of the active window 220 as the disturbing image. Note that the information concealment system 100 may execute the concealment processing using an image of an area that includes the work area 224 of the active window 220 and is larger than the work area 224 as a concealment image.

In the present embodiment, an example of the information concealment system 100 has been described, taking as an example the case where the information displayed in the active window 220 is concealed. However, the information concealment system 100 is not limited to this embodiment. In other embodiments, information displayed in inactive window 240 may be masked. In this case, the information displayed in the inactive window 240 may be an example of confidential information.

In this embodiment, the case where a single output frame 302 comprises a single sub-frame 312 corresponding to the original image and a single sub-frame 314 corresponding to the occulting image is taken as an example. An example was described. However, the information concealment system 100 is not limited to this embodiment. In other embodiments, a single output frame may comprise 3 or more subframes, or 4 or more subframes. In a single output frame, the number of subframes corresponding to the original image and the number of subframes corresponding to the occultation image are preferably the same.

In this embodiment, in a single output frame 302, a single sub-frame 312 corresponding to the original image and a single sub-frame 314 corresponding to the occultation image are arranged in this order. , an example of the information concealment system 100 has been described. However, the information concealment system 100 is not limited to this embodiment. In other embodiments, the positions in subframes in a single output frame may be arbitrarily determined. For example, a single sub-frame 314 corresponding to the occultation image is followed by a single sub-frame 312 corresponding to the original image.

Another example of anonymization processing in the information anonymization system 100 will be described with reference to FIGS. 6 and 7. FIG. FIG. 6 schematically illustrates an example output frame 602 containing data for an image displayed on display 122 . FIG. 5 schematically shows an example desktop image 700 as perceived by third party 32 .

As shown in FIG. 6, output frame 602 includes subframe 612 , subframe 614 , subframe 616 , and subframe 618 . Display terminal 120 processes subframe 612 to display desktop image 320 in desktop display area 212 of display 122 . Similarly, display terminal 120 processes sub-frame 614 to cause desktop image 660 to be displayed in desktop display area 212 of display 122 . Display terminal 120 processes subframe 616 to display desktop image 340 in desktop display area 212 of display 122 . Display terminal 120 processes sub-frame 618 to display desktop image 680 in desktop display area 212 of display 122 .

In this embodiment, desktop image 660 may be the same image as desktop image 320 except that dummy image 662 is displayed instead of window image 322 . Similarly, desktop image 680 may be the same image as desktop image 320 except that dummy image 682 is displayed in place of window image 322 . Dummy image 662 and dummy image 682 may be the same image, different images that make up a single moving image, or different images that make up a series of images output from a single application program. There may be.

According to this embodiment, display terminal 120 processes a plurality of output frames having a configuration similar to output frame 602 so that display 122 displays desktop image 320 , desktop image 660 , desktop image 340 and desktop image 680 . are displayed in time division. On the other hand, in the present embodiment, the eyewear device 140 (i) opens the optical shutter 142 while the desktop image 320 is displayed on the display 122 , and (ii) displays the desktop image 660 and the desktop image 340 on the display 122 . And during the period when the desktop image 680 is displayed, the opening and closing of the optical shutter 142 is controlled so that the optical shutter 142 is open.

As shown in FIG. 7 , a region of desktop image 700 corresponding to active window 220 of desktop image 200 appears to third person 32 to be obscured by fusion image 724 . When the window image 342 is a complementary color image of the window image 322, the fusion image 724 is an image in which the gray image and the dummy image are mixed. Thereby, the anonymization process can be executed without being noticed by the third party 32 . On the other hand, the user 20 wearing the eyewear device 140 can visually recognize the desktop image 200 .

[An example of another embodiment]
In this embodiment, the case where the output frame 602 comprises a sub-frame 612 corresponding to the original image, a sub-frame 616 corresponding to the occultation image, and sub-frames 614 and 618 corresponding to the dummy image is taken as an example. , an example of an information concealment system 100 has been described. However, the information concealment system 100 is not limited to this embodiment. In another embodiment, the output frame comprises one or more sub-frames corresponding to the original image, one or more sub-frames corresponding to the occultation image, and one or more sub-frames corresponding to the dummy image. The order of the sub-frames corresponding to the original image, the sub-frames corresponding to the occultation image, and the sub-frames corresponding to the dummy image in the output frame can be arbitrarily determined.

FIG. 8 schematically shows an example of the internal configuration of the display terminal 120. As shown in FIG. In this embodiment, the display terminal 120 includes an input section 820 , a display 122 , a communication section 840 and a control section 860 . In this embodiment, the control unit 860 has a peripheral device management unit 862 , an application execution unit 863 , an image display control unit 864 , an optical shutter control unit 865 and a communication control unit 866 .

In this embodiment, the input unit 820 receives input from the user 20 . Examples of the input unit 820 include input devices such as a keyboard, pointing device, and camera. Examples of pointing devices include a mouse and a touch panel. Note that when the input unit 820 includes a camera, devices existing around the display terminal 120 can be detected based on the image captured by the camera.

The input unit 820 may receive instructions from the user 20 to the display terminal 120 . For example, input unit 820 receives an instruction to change at least one of the position, shape, and size of the window (sometimes referred to as a change instruction).

In this embodiment, the communication unit 840 transmits and receives information to and from devices external to the display terminal 120 . The communication unit 840 may transmit/receive information to/from an external device using a wired communication method, or may transmit/receive information to/from an external device using a wireless communication method. For example, the communication unit 840 is configured to communicate with at least one of the one or more eyewear-type devices 140 . The communication unit 840 may be configured to wirelessly communicate with at least one of the one or more eyewear-type devices 140 . The wireless communication method may be a communication method using radio waves, a communication method using infrared rays, or a communication method using visible light. The wireless communication method may be WiFi (registered trademark) or Bluetooth (registered trademark).

For example, the communication unit 840 transmits and receives various control signals to and from each of the one or more eyewear type devices 140 . The communication unit 840 may receive a signal including an ID unique to each device from one or more devices existing around the display terminal 120 . The unique ID is exemplified by MAC address, Bluetooth (registered trademark) Device ID (sometimes referred to as UUID), and the like. A unique ID may be some distinguishable information stored or stored in a particular item. Note that the peripheral device can be detected based on the communication unit 840 receiving the unique ID.

If the communication unit 840 is configured to be able to transmit and receive information to and from an external device using a wireless communication method, the communication unit 840 may be configured to be able to adjust the strength of the transmission radio wave. For example, when the distance between the user 20 and the display terminal 120 is smaller than the distance between the third party 34 and the display terminal 120, the communication unit 840 selectively sends a control signal to the eyewear device 140 worn by the user 20. can be sent. If the radio wave intensity of the radio wave for transmitting the control signal is low, the eyewear device 150 worn by the third party 34 cannot receive the control signal. Peeping by the third party 34 can thereby be suppressed.

In this embodiment, the controller 860 controls the operation of the display terminal 120 . The control section 860 may control the operation of the eyewear-type device 140 by transmitting control signals to the eyewear-type device 140 .

In this embodiment, the peripheral device management unit 862 manages various devices existing around the display terminal 120 . The peripheral device management unit 862 manages devices (sometimes referred to as peripheral devices) that have at least one of an optical shutter and an image pickup device among peripheral devices.

The peripheral device management unit 862 may manage an ID unique to each of one or more peripheral devices. Peripheral device manager 862 may manage the state of each of one or more peripheral devices. The state of the peripheral device is exemplified by the remaining battery level. The peripheral device management unit 862 may manage the presence or absence of information regarding a specific item among a plurality of items regarding the state of the peripheral device.

The peripheral device management unit 862 may manage whether or not each of the one or more peripheral devices is a device that meets the genuine product determination condition (sometimes referred to as a regular device). The genuine product determination condition may be a condition related to a device that is permitted to perform confidential information recognizable processing.

The conditions for judging genuine products include the condition that the ID unique to the peripheral device is registered in the whitelist, the condition that the ID unique to the peripheral device is not registered in the blacklist, and the specifications of the peripheral device are specified. a condition that information on a specific item related to the state of the peripheral device is acquired, and a predetermined character string in the control signal transmitted from the peripheral device to the display terminal 120 For example, a condition that the information indicating is included is included. When information about a particular item regarding the status of a peripheral device is obtained, it can be determined that the specifications of the peripheral device meet a particular condition. For example, if information regarding the remaining battery level of a peripheral device has been obtained, it may be determined that the peripheral device has a specification that allows transmission of information regarding the remaining battery level.

In this embodiment, the application execution unit 863 executes application programs. Application execution unit 863 outputs a signal (sometimes referred to as an input image signal) for displaying the execution result of the application program in the work area of the window corresponding to the application program.

In this embodiment, the image display control section 864 controls the display 122 . The image display control section 864 may control the content of the image or information displayed on the display 122 by controlling the display 122 .

In one embodiment, the image display control unit 864 identifies confidential information and performs anonymization processing for the confidential information. The image display control unit 864 may determine whether or not the anonymization process is necessary for each window. The image display control section 864 may control the display 122 so that the display 122 displays the original image and the obscuration image in a time division manner for each frame.

In another embodiment, when a device that does not meet the above-described genuine product determination condition (sometimes referred to as a suspicious device) is detected during execution of anonymization processing, the image display control unit 864 The above-described second anonymizing process is executed. As described above, the image display control section 864 may determine the details of the process to be executed based on the level of risk of information leakage. For example, the image display control unit 864 stops displaying confidential information or an image containing the confidential information (sometimes referred to as a confidential image). If a suspicious device is detected during execution of anonymization processing, the image display control section 864 may change the display timing of the confidential information or the confidential image.

When the image display control unit 864 changes the display timing of the confidential information or the confidential image, the image display control unit 864 transmits a signal including the information indicating the change or the information indicating the display timing after the change to the eyewear. may be sent to type device 140 . As described above, the image display control section 864 may encrypt the information indicating the change or the information indicating the display timing after the change, and transmit the encrypted information to the eyewear device 140.

Specifically, for example, the image display control unit 864 first determines whether the one or more peripheral devices detected by the peripheral device management unit 862 include a suspicious device. If it is determined that the one or more peripheral devices do not include a suspicious device (sometimes referred to as a case where no suspicious device is detected), the image display control unit 864 displays the original image and the obscuration image. , to be displayed on the display 122 in a time division manner. Thereby, anonymization processing is executed. On the other hand, if it is determined that a suspicious device is included in one or more peripheral devices (sometimes referred to as a case where a suspicious device is detected), image display control section 864 performs the above-described second Decide to execute the anonymization process. For example, the image display control unit 864 decides to stop displaying the confidential information or the confidential image, or to change the display timing of the confidential information or the confidential image. Thereby, for example, peeping by a third party wearing the eyewear type device 150 is suppressed.

In this embodiment, the optical shutter controller 865 controls the operation of the optical shutter 142 of the eyewear device 140 . The optical shutter control section 865 may control the operation of the optical shutter 142 of the eyewear type device 140 by transmitting a control signal for controlling the operation of the optical shutter 142 of the eyewear type device 140 to the eyewear type device 140 .

For example, the optical shutter control section 865 controls the opening/closing timing of the optical shutter 142 . The optical shutter control unit 865 transmits to the eyewear device 140 a control signal indicating the length of the opening/closing period Pg of the optical shutter 142 and the position of the period Pgo during which the optical shutter 142 is open in the opening/closing period Pg of the optical shutter 142. By doing so, the opening/closing timing of the optical shutter 142 may be controlled. The above control signal may include information indicating the length of the period Pgo during which the optical shutter 142 is open, and may include information indicating the length of the period Pgc during which the optical shutter 142 is closed.

The above control signal may include a synchronization signal for synchronizing the switching of images on the display 122 with the opening and closing of the optical shutters 142 on the eyewear type device 140 . The above control signal may include information indicating that a suspicious device has been detected, may include information indicating that display of the confidential image on display 122 is to be stopped, and display timing of the confidential image on display 122 may be It may contain information indicating that it is changed.

The optical shutter control section 865 may control the operation of the optical shutter 142 based on information from the image display control section 864 . For example, the optical shutter controller 865 generates the above control signal based on information from the image display controller 864 .

The optical shutter control section 865 may acquire information indicating the length of the frame period Pf from the image display control section 864 and determine the length of the opening/closing period Pg of the optical shutter 142 based on the length of Pf. The optical shutter control unit 865 acquires information indicating the position of the period Pso during which the sub-frame 312 is processed in the frame period Pf from the image display control unit 864, and controls the optical shutter 142 based on the position of the period Pso in the frame period Pf. may determine the position of the period Pgo in the opening and closing period Pg of . The optical shutter control section 865 may acquire information indicating the length of the period Pso from the image display control section 864, and determine the length of the opening/closing period Pg of the optical shutter 142 based on the length of Pso. The optical shutter control section 865 may acquire information indicating the length of the period Psj from the image display control section 864, and determine the length of the opening/closing cycle Pg of the optical shutter 142 based on the length of Psj.

In this embodiment, the communication control section 866 controls the communication section 840 . The communication control unit 866 may support a wired communication system, or may support one or more wireless communication systems. The communication control section 866 may control transmission and reception of information between the eyewear device 140 and each section of the control section 860 . For example, the communication control unit 866 controls the communication unit 840 so that the control signal from the optical shutter control unit 865 is transmitted to the specified eyewear device 140 or unspecified eyewear device 140. .

According to one embodiment, a control signal including a synchronization signal is sent to the eyewear device 140 when no suspicious device is detected. According to another embodiment, a control signal is sent to the eyewear device 140 when a suspicious device is detected and the image display control section 864 decides to change the display timing. The control signal includes, for example, information indicating that the display timing is changed, information indicating the display timing after the change, and the like. The information indicating the changed opening/closing timing may be an example of information indicating the changed display timing.

The communication control section 866 may control the radio wave intensity of the transmission radio wave output by the communication section 840 . For example, when a suspicious device is detected and the image display control unit 864 decides to change the display timing, the communication control unit 866 controls the radio wave intensity of the radio wave for transmitting the control signal so that it becomes weaker than the predetermined intensity. Then, the communication unit 840 is controlled.

Input unit 820 may be an example of an instruction receiving unit. The camera of the input unit 820 or part thereof may be an example of an imaging device. Communication unit 840 may be an example of a transmitter. Control unit 860 may be an example of a control device. The peripheral device manager 862 may be an example of a peripheral device detector. The image display control section 864 may be an example of a display control section or a determination section. The communication control section 866 may be an example of a transmission control section.

FIG. 9 schematically shows an example of the internal configuration of the image display control section 864. As shown in FIG. In this embodiment, an example of the image display control unit 864 will be described by taking as an example a case where a complementary color image is generated as an obscuration image.

In this embodiment, the image display control unit 864 includes, for example, a user input reception unit 922, a pointer position determination unit 924, and a window management unit 926. In this embodiment, the image display control unit 864 includes, for example, an input image generation unit 932, an input image buffer 934, a complementary color image generation unit 936, a complementary color image buffer 938, an output image generation unit 940, and an image output management unit. and a section 950 .

In this embodiment, the user input reception unit 922 receives input from the user 20 . Specifically, user input reception unit 922 acquires information indicating the content of the user input input to input unit 820 . For example, the user input accepting unit 922 accepts an instruction to change at least one of the position, shape and size of the window.

In this embodiment, pointer position determiner 924 determines the position of pointer 260 on display 122 . The pointer position determining section 924 may determine the position of the pointer 260 on the display 122 based on the user input from the pointing device received by the user input receiving section 922 .

In this embodiment, window manager 926 manages each of the one or more windows displayed in desktop display area 212 of display 122 . The window manager 926 may manage information regarding the state of each of the one or more windows. Details of the window state information will be described later.

In this embodiment, the input image generation unit 932 generates an input image that is the original image. Specifically, based on the input image signal from application execution unit 863, input image generation unit 932 generates the desktop image displayed in desktop display area 212 as the input image. As mentioned above, the desktop image may include images of one or more windows. The input image generator 932 stores the generated desktop image in the input image buffer 934 .

In this embodiment, the complementary color image generator 936 generates a complementary color image that is an example of an obscuration image. In this embodiment, the case where the complementary color image generation unit 936 generates a complementary color image having a complementary color relationship with an image forming at least a part of the input image generated by the input image generation unit 932 is taken as an example. An example of is described.

For example, the complementary color image generating unit 936 causes the desktop image including the complementary color image of the image of the window containing the confidential information among the one or more windows arranged on the desktop to correspond to the desktop image generated by the input image generating unit 932. Generate as an occultation image. The complementary color image generation section 936 stores the generated desktop image in the complementary color image generation section 936 .

Specifically, the complementary color image generator 936 first acquires the desktop image generated by the input image generator 932 . Further, the complementary color image generation unit 936 identifies a target window to be subjected to anonymization processing among the one or more windows. For example, the complementary color image generating section 936 identifies the target window based on the user input received by the user input receiving section 922 . The complementary color image generator 936 may identify one or more windows corresponding to a specific type of application program as target windows.

Next, the complementary color image generation unit 936 refers to information managed by the window management unit 926 and obtains information indicating the position and size of the target window (sometimes referred to as window information). The complementary color image generation unit 936 may refer to information managed by the window management unit 926 to obtain information indicating the shape of the target window.

Next, the complementary color image generation unit 936 identifies a target area for complementary color image generation processing in the desktop image generated by the input image generation unit 932 based on the position, shape and size of the target window. This determines the position, shape and size of the area where the complementary color image is generated.

The area to be subjected to the complementary color image generation process may be the same as the area in which the target window is arranged, or may be larger than the area in which the target window is arranged. The target area of the complementary color image generation process may be an area including at least part of the area in which the target window is arranged.

In one embodiment, the complementary color image generator 936 generates (i) an area in which the complementary color image of the target window is displayed when the complementary color image of the target window is displayed on the display 122 in the desktop display area 212 of the display 122 ( (ii) an area where the original image of the target window is displayed when the original image of the target window is displayed on the display 122 (sometimes referred to as a display area); ), the location, shape and size of the region in which the complementary color image is generated may be determined. Thereby, leakage of confidential information can be further suppressed.

In another embodiment, the complementary color image generator 936 determines the position, shape and size of the complementary color image of the target window according to the speed at which at least one of the position, shape and size of the target window is changed. good too. For example, the complementary color image generator 936 adjusts the degree of superimposition of the disturbance area and the display area according to the speed. The complementary color image generator 936 may determine the position, shape, and size of the complementary color image of the target window such that the degree of overlapping of the obstruction area and the display area increases as the speed increases.

Next, the complementary color image generation unit 936 converts at least one of the color tone and luminance of each pixel arranged in the area where the complementary color image is generated in the desktop image generated by the input image generation unit 932 into an image having a complementary color relationship. is converted based on a known algorithm for generating . As a result, for example, a desktop image including a complementary color image of the target window is generated.

For example, the complementary color image generation unit 936 first acquires or generates information indicating the correspondence relationship between the colors of the original image and the colors of the complementary color image. The correspondence relationship may be the first complementary color relationship described above or the second complementary color relationship. The above correspondence relationship may be represented by a function or by a table.

Next, the complementary color image generation unit 936 generates a complementary color image of the target window by converting the color of each pixel of the target window of the original image based on the correspondence relationship between the colors of the original image and the colors of the complementary color image. do. Also, the complementary color image generator 936 synthesizes the complementary color image of the target window at an appropriate position of the original image of the desktop image. As a result, a desktop image including a complementary color image of the target window is generated.

Note that the complementary color image generator 936 may adjust the lightness or brightness of at least a portion of one or more pixels forming the original image of the desktop image, and then perform the process of synthesizing the complementary color image of the target window. For example, the complementary color image generation unit 936 synthesizes an image in which the brightness or brightness of the entire original image or at least part of the pixels in the area other than the target window is smaller than that of the original image, and the complementary color image of the target window. you can

In this embodiment, the output image generator 940 generates an image displayed on the display 122 (sometimes referred to as an output image). The output image generator 940 may generate an output image based on the control signal from the image output manager 950 . For example, the output image generator 940 generates an output image based on a control signal indicating display timing.

Specifically, the output image generator 940 outputs the desktop image stored in the input image buffer 934 at the timing when the original image is displayed. Also, the output image generation unit 940 outputs the desktop image stored in the complementary color image buffer 938 at the timing when the obscuration image is output.

[Adjustment of the output image considering the display characteristics of the display]
The output image generator 940 may correct the desktop image stored in the input image buffer 934 in consideration of the display characteristics of the display 122 and output the corrected image to the display 122 . The output image generator 940 may correct the desktop image stored in the complementary color image buffer 938 in consideration of the display characteristics of the display 122 and output the corrected image to the display 122 .

[Adjustment of the output image considering the pointer position]
The output image generator 940 may correct the desktop image stored in the complementary color image buffer 938 when the pointer 260 is positioned near or inside the complementary color image of the target window. Also, the output image generation section 940 may output the corrected image to the display 122 .

Specifically, the output image generation unit 940 first acquires information indicating the position of the pointer 260 on the display 122 from the pointer position determination unit 924 . In addition, the output image generation unit 940, for example, accesses the OS and acquires information regarding colors treated as key colors. Next, the output image generator 940 determines whether the pointer 260 is positioned near or inside the complementary color image of the target window. The neighborhood definition is determined, for example, by system settings or user designation.

If it is determined that the pointer 260 is not positioned near or within the complementary color image of the target window, the output image generator 940 may, for example, display the desktop image stored in the complementary color image buffer 938 on the display 122 without correcting it. Output. On the other hand, if the pointer 260 is determined to be near or within the complementary color image of the target window, the output image generator 940 determines to correct the desktop image stored in the complementary color image buffer 938 . The output image generation unit 940 changes the color of the area including the indicated position of the pointer 260 (sometimes referred to as a pointer area) to the key color in the desktop image. As a result, even when the corrected desktop image is displayed in the foremost active window, the user 20 can select or operate the icon or window arranged behind the active window.

The shape and size of the pointer area are determined, for example, by system settings or user designation. The shape and size of the pointer area may be determined based on at least one of the direction of movement of the pointer 260, the speed of movement of the pointer 260, the type of pointing device, and the setting of the pointing device.

For example, the shape and size of the pointer area are determined such that the length of the pointer area in the moving direction of the pointer 260 is greater than the length of the pointer area in the direction opposite to the moving direction of the pointer 260 . For example, the shape and size of the pointer area are determined so that the size of the pointer area increases as the moving speed of the pointer 260 increases. For example, when the type of pointing device is a touch panel, the shape and size of the pointer area are determined so that the size of the pointer area is larger than when the type of pointing device is a mouse. For example, the shape and size of the pointer area may be determined such that the greater the sensitivity of the pointing device, the larger the size of the pointer area.

In this embodiment, the image output management unit 950 manages image output. Details of the image output management unit 950 will be described later.

User input reception unit 922 may be an example of an instruction reception unit. The input image generator 932 may be an example of a confidential image generator. The complementary color image generation unit 936 may be an example of an interference image generation unit, a target identification unit, or a window information acquisition unit. The output image generator 940 may be an example of a confidential image generator or a disturbing image generator. The image output management section 950 may be an example of a determination section. An image of a window containing confidential information may be an example of a confidential image. A region in which a complementary color image is generated may be an example of a hidden region.

The pointer position determiner 924 may be an example of a pointer detector. The complementary color image generating section 936 may be an example of a hidden area determining section, a position specifying section, or a range determining section. The output image generation section 940 may be an example of a color information acquisition section or an interference image generation section.

[An example of another embodiment]
In this embodiment, when it is determined that the pointer 260 is located near or inside the complementary color image of the target window, the output image generation unit 940 corrects the desktop image stored in the complementary color image buffer 938. As an example, one example of the information concealment system 100 has been described. However, the information concealment system 100 is not limited to this embodiment.

In another embodiment, a complementary color image is generated in which the color of the pointer area is set to the key color regardless of the position of pointer 260 . For example, the complementary color image generation unit 936 generates a complementary color image in which the color of the pointer area is set as the key color. This allows the pointer to pass through the active window even when the complementary color image is displayed in the active window.

FIG. 10 schematically shows an example of the data table 1000. As shown in FIG. In this embodiment, the data table 1000 includes, for each of one or more windows, a window ID 1022, title name information 1024, process ID 1032, process status information 1034, layout information 1042, and transparency information 1042. Information 1044 indicating settings related to encryption and information 1052 indicating necessity of anonymization are stored in association with each other.

Examples of the information 1042 regarding the layout include the position, shape, size, superimposition relationship with other windows, and the like. If the shape of the window is predetermined by initial settings, user settings, or the like, information about the shape of the window may not be stored in the data table 1000 . The order of superimposition is exemplified as the superimposition relationship with other windows.

Non-transparency, semi-transparency, and transparency are exemplified as settings related to transparency. The setting for transparency may be the setting for transparent color or transparent color described above. Also, if a single window contains two or more areas with different transparency settings, the identification information of each area, information on the position, shape and size of each area, and the transparency of each area may be stored in association with information indicating settings related to.

According to this embodiment, for example, the window with the title "word_contract.doc" corresponding to the process P002 is anonymized according to an instruction from the user. When the display terminal 120 receives the above instruction, the process P011 of anonymization processing is executed to generate a new window. The generated new window is placed, for example, in the foreground. The title name of the new window may include part of the title name of the target window to be anonymized. Note that the title name of the new window may be the same as the title name of the target window to be anonymized. Alternatively, the target may be determined by performing a fuzzy search process using a filter that defines the anonymization target in advance using a regular expression.

FIG. 11 schematically shows an example of the internal configuration of the output image generator 940. As shown in FIG. In this embodiment, the output image generator 940 includes a display window generator 1122 , an obscuration window generator 1124 , a superimposition state determiner 1132 , an image corrector 1134 and an image combiner 1140 .

In this embodiment, the display window generation unit 1122 generates a window for displaying information presented to the user 20 (sometimes referred to as a display window). The display window generation unit 1122 may generate one or more display windows corresponding to one or more application programs executed by the application execution unit 863, respectively.

In this embodiment, the occlusion window generator 1124 generates at least a window for displaying an occultation image (sometimes referred to as an occlusion window). In one embodiment, the occultation window displays the original image and the occultation image in a time division manner. In another embodiment, the obscuration window displays the transparent image and the obscuration image in a time division manner. In yet another embodiment, an occultation image is selectively displayed in the occlusion window. The details of the image displayed in the occultation window are described later.

In this embodiment, the overlay state determination unit 1132 determines the overlay state of one or more display windows and obscuration windows. Details of the superimposed state will be described later.

In this embodiment, the image correction unit 1134 corrects at least one of the desktop image stored in the input image buffer 934 and the desktop image stored in the complementary color image buffer 938 . In one embodiment, image corrector 1134 corrects the desktop image stored in complementary color image buffer 938 to take into account the display characteristics of display 122 . For example, the image corrector 1134 corrects the desktop image stored in the complementary color image buffer 938 by reducing the color tone range. This suppresses the occurrence of shading in the fusion image generated by fusing the original image and the obscuration image.

In another embodiment, when the desktop image stored in the input image buffer 934 is switched at a display speed of 15 frames per second or more and displayed on the display 122, the desktop image is displayed with an afterimage. generates a first image and a second image of . The first image and the second image may be, for example, images in which corresponding areas differ in color tone or brightness.

Also, the image correction unit 1134 generates an obscuration image corresponding to the first image and an obscuration image corresponding to the second image. The image corrector 1134 may generate an obscuration image corresponding to the first image and an obscuration image corresponding to the second image based on the desktop image stored in the complementary color image buffer 938 . The obscuration image corresponding to the first image and the obscuration image corresponding to the second image may be, for example, images in which the corresponding regions have different color tones or brightness.

In this embodiment, the image synthesizing unit 1140 synthesizes the image displayed in each of the one or more display windows and the image displayed in the obscuration window to generate the output image. The image synthesizer 1140 outputs the generated image to the display 122 at the timing indicated by the control signal from the image output manager 950 . Thereby, the image displayed by the display 122 is controlled.

In one embodiment, the obscuring window may be a full-screen window that substantially entirely covers desktop viewing area 212 of display 122 . The obscuration window may be arranged in front of one or more display windows. In this case, for example, at the timing when the original image is displayed on the display 122, the image synthesizing unit 1140 outputs the output image so that the obscuration window in which the original image is displayed is arranged in front of one or more display windows. to generate Further, the image synthesizing unit 1140 generates an output image such that the occlusion window displaying the occlusion image is arranged in front of one or more display windows at the timing when the occlusion image is displayed on the display 122. .

In other embodiments, the obscuring window may be a full screen window that substantially entirely covers the desktop viewing area 212 of the display 122 and has transparent or transparent areas. The obscuration window may be arranged in front of one or more display windows. In this case, for example, at the timing when the original image is displayed on the display 122 , the obscuration window having a transparent region or a transparent region at least in the region where the target window is displayed is larger than one or more display windows. Generate the output image so that it is placed in the foreground. Further, the image synthesizing unit 1140 generates an output image such that the occlusion window displaying the occlusion image is arranged in front of one or more display windows at the timing when the occlusion image is displayed on the display 122. .

In yet another embodiment, the image synthesizing unit 1140 changes the superimposing order of the obscuration window and the target window according to the display timing, thereby executing the anonymization process. The image synthesizing unit 1140 generates the output image so that the target window is positioned in front of the obscuration window at the timing when the original image is displayed on the display 122, for example. Further, the image synthesizing unit 1140 generates an output image such that the obscuration window is arranged in front of the target window at the timing when the obscuration image is displayed on the display 122 .

An occultation window may be an example of an obstruction window. The image synthesizing unit 1140 may be an example of a confidential image generating unit or a disturbing image generating unit.

FIG. 12 schematically shows an example of a superimposed state of the image 1200. As shown in FIG. In this embodiment, image 1200 includes background layer 1210 , window layer 1220 and pointer layer 1230 . In this embodiment, window layer 1220 includes one or more display windows 1222 and obscuration windows 1224 . Display windows 1222 include, for example, active window 220 . Display windows 1222 may include an active window 220 and one or more inactive windows 240 .

As described in connection with FIG. 11, in one embodiment, the occlusion window 1224 displays the original image and the occultation image in a time split manner. In another embodiment, the occlusion window 1224 displays the transparency image and the occlusion image in a time division manner. Background layer 1210 and one or more display windows 1222 are displayed on display 122 during periods when a transparent image is displayed in obscuration window 1224 . As a result, the original image is displayed on the display 122 .

The display method of the transparent image is not particularly limited, but the transparent image can be displayed on the display 122 by the following procedure, for example. First, for example, the OS of the display terminal 120 sets the transparent color (sometimes referred to as transparent color) of the form such as the obscure window 1224 . The transparent color is set by the TransparencyKey property, for example. The display terminal 120 generates a monochromatic image of the color set as the transparent color at the timing when the original image is displayed, and causes the obscuration window 1224 to display the monochromatic image. As a result, the above monochromatic image is treated as a transparent image.

FIG. 13 schematically shows an example of correction processing by the image correction unit 1134. FIG. In this embodiment, output frame 1302 has subframe 1322 , subframe 1324 , subframe 1326 , and subframe 1328 .

Display terminal 120 processes subframe 1322 to display desktop image 1342 in desktop display area 212 of display 122 . Similarly, display terminal 120 processes sub-frame 1324 to cause desktop image 1344 to be displayed in desktop display area 212 of display 122 . Display terminal 120 processes subframe 1326 to display desktop image 1346 in desktop display area 212 of display 122 . Display terminal 120 processes subframe 1328 to display desktop image 1348 in desktop display area 212 of display 122 .

For example, when the desktop image 1342 and the desktop image 1346 are switched at a display speed of 15 frames per second or more and displayed on the display 122, the afterimage generates the original image to be displayed in the output frame 1302. At least one of the color tone and brightness of the pixels forming each image is adjusted as described above. For example, the color tone and brightness of each pixel making up desktop image 1346 is determined based on the color tone and brightness of the corresponding pixel in desktop image 1342 .

Referring to FIG. 13, the correspondence between the pixels of the desktop image 1342 and the desktop image 1346 will be described by taking the area surrounded by the dotted line of the desktop image 1342 and the desktop image 1346 as an example. In this embodiment, the brightness of each pixel is adjusted out of the color tone and brightness of each pixel. For example, the brightness of one or more pixels included in each of regions 1362 and 1368 is set higher than the brightness of the original image in desktop image 1342 and lower than the brightness of the original image in desktop image 1346. . Similarly, one or more luminances included in each of regions 1364 and 1366 are set lower than the luminance of the original image in desktop image 1342 and higher than the luminance of the original image in desktop image 1346 .

Note that, in this embodiment, the desktop image 1344 indicates a complementary color image of the desktop image 1342 . Desktop image 1348 shows a complementary color image of desktop image 1346 .

[An example of another embodiment]
In the present embodiment, each of the regions 1362, 1364, 1366, and 1368 has a rectangular shape, the shape of each region is constant regardless of time, and the mode of correction for at least one of color tone and luminance is An example of the information concealment system 100 has been described, taking as an example a case where the information is constant irrespective of time. However, the information concealment system 100 is not limited to this embodiment.

In other embodiments, the shape of each region described above may be any shape. In other embodiments, the shape of each of the above regions may vary over time. That is, the shape of each region in one output frame may differ from the shape of each region in another output frame. In yet another embodiment, the aspect of the tone and/or luminance correction in each region may vary over time. That is, the mode of correction of each area in one output frame may be different from the mode of correction of each area in another output frame.

As a result, when the display pattern of the image on the display 122 and the opening/closing pattern of the optical shutter 142 are out of sync, the area (in the example of FIG. 1364, 1366, and 1368 are treated as one unit, and image correction processing is performed.) appears to flow vertically and horizontally or rotate over time. Also, when the amount of luminance correction varies from frame to frame, the area that is the unit of correction processing appears to flicker or the amount of light in the area increases or decreases. As a result, information leakage is suppressed.

An example of composition processing by the image composition unit 1140 will be described with reference to FIGS. 14, 15, and 16. FIG. According to this embodiment, the desktop image 320 is displayed in the desktop display area 212 of the display 122 at the display timing of the original image, and the desktop image 340 is displayed in the desktop display area 212 of the display 122 at the display timing of the obscuration image. An example of the composition processing by the image composition unit 1140 will be described by taking a case where the images are combined as an example.

FIG. 14 schematically shows an example of synthesizing processing by the image synthesizing unit 1140 . FIG. 15 schematically shows another example of synthesizing processing by the image synthesizing unit 1140. In FIG. FIG. 16 schematically shows still another example of composition processing by the image composition unit 1140. In FIG.

In the embodiment described in connection with FIGS. 14 and 15, in the window layer 1220, the obscuration window is placed on top. Also, the obscuration window is placed in the foreground of the window layer 1220 while the anonymization process is being performed. On the other hand, in the embodiment described in connection with FIG. 16, the overlapping order of the obscuration windows in the window layer 1220 fluctuates while the concealment processing is being performed.

In the embodiment described with reference to FIG. 14, the image synthesizing unit 1140 is configured so that the desktop image 1420 serving as the original image and the desktop image 1440 serving as the obscuration image are displayed in a time division manner in the obscuration window. Control the display 122 . Specifically, the image synthesizing unit 1140 controls the display 122 by outputting an output image signal for controlling the image displayed on the display 122 .

As shown in FIG. 14, desktop image 1420 may be a composite image of opaque background image 1410, window image 322, and window image 324. FIG. The background image 1410 can be generated, for example, by capturing a background image before execution of the anonymization process is started or immediately after the execution of the anonymization process is started. In one embodiment, desktop image 1420 may be an image synthesized by input image generator 932 and stored in input image buffer 934 . In other embodiments, desktop image 1420 may be synthesized by output image generator 940 .

As shown in FIG. 14, desktop image 1440 may be a composite image of opaque background image 1410, window image 342, and window image 344. FIG. In one embodiment, desktop image 1440 may be an image synthesized by complementary image generator 936 and stored in complementary image buffer 938 . In other embodiments, desktop image 1440 may be synthesized by output image generator 940 .

According to this embodiment, a desktop image 1420 and a desktop image 1440 having an opaque background image 1410 are displayed in obscuring windows arranged in front of the one or more display windows so that the one or more display windows are Covered up. Also, for example, desktop image 1420 and desktop image 1440 are time-divided to be displayed in the obscuration window by processing the multiple output frames described in connection with FIG. As a result, the image of the target window is made anonymous.

In the embodiment described with reference to FIG. 15, the image synthesizing unit 1140 controls the display 122 so that the obscuration window displays a desktop image 1520 and a desktop image 1540 serving as the obscuration image in a time division manner. do. Specifically, the image synthesizing unit 1140 controls the display 122 by outputting an output image signal for controlling the image displayed on the display 122 .

As shown in FIG. 15, in this embodiment the desktop image 1520 has a transparent background image 1510 . The desktop image 1520 is generated by the output image generator 940, for example. By displaying the desktop image 1520 in the obscuration window arranged in front of the one or more display windows, the original image of the one or more display windows and the original image of the background image are displayed in the desktop display area of the display 122. 212.

As shown in FIG. 15, in this embodiment, desktop image 1540 may be an image in which transparent background image 1510 and window image 342 are combined. The arrangement of the window image 342 in the desktop image 1540 is such that, for example, the area in which the window image 322 is arranged in the desktop image 320 that is the original image and the area in which the window image 342 is arranged in the desktop image 1540 that is the obscuration image are approximately equal to each other. determined to match. The desktop image 1540 is generated, for example, by the output image generator 940 combining the transparent background image 1510 and the window image 342 generated by the output image generator 940 and stored in the complementary color image buffer 938 . .

According to this embodiment, the desktop image 1540 is displayed in the obscuring window arranged in front of one or more display windows, thereby obscuring the window image 322 to be anonymized. On the other hand, since the background image 1510 of the desktop image 1540 is transparent, the display window images other than the window image 322 are displayed in the desktop display area 212 of the display 122 .

In the embodiment described with reference to FIG. 16, the image synthesizing unit 1140 is configured such that the active window 220 to be anonymized is placed in front of the obscure window 1224 on which at least the window image 342 is displayed. The display 122 is arranged so that the image 1620 and the desktop image 1640 in which the obscuration window 1224 displaying at least the window image 342 is placed in front of the active window 220 to be anonymized are displayed in a time division manner. to control. Specifically, the image synthesizing unit 1140 controls the display 122 by outputting an output image signal for controlling the image displayed on the display 122 .

As shown in FIG. 16, the position, shape and size of obscuring window 1224 in which at least window image 342 is displayed may substantially match the position, shape and size of active window 220 . The arrangement of the window image 342 in the desktop image 1640 is such that, for example, the area in which the window image 322 is arranged in the desktop image 320 that is the original image and the area in which the window image 342 is arranged in the desktop image 1640 that is the obscuration image are approximately equal to each other. determined to match.

In another embodiment, the position, shape, and size of the obscuring window 1224 are such that when the obscuring window 1224 is placed in front of the active window 220, the obscuring window 1224 covers the entire active window 220. may be determined as In still other embodiments, obscuring window 1224 may be a full screen window that substantially entirely covers desktop viewing area 212 of display 122 . In this case, obscuration window 1224 may display desktop image 1440 or desktop image 1540 .

Desktop image 1420 may be an example of a hidden image. Desktop image 1440 may be an example of a disturbing image. Desktop image 1520 may be an example of a hidden image. Desktop image 1540 may be an example of a disturbing image. Desktop image 1620 may be an example of a hidden image. Desktop image 1640 may be an example of a disturbing image.

FIG. 17 schematically shows an example of the internal configuration of the image output management section 950. As shown in FIG. In this embodiment, the image output management section 950 includes a management information storage section 1720 , a display timing determination section 1732 , an open/close timing determination section 1734 , a genuine product determination section 1740 , and a control signal output section 1750 . In this embodiment, the management information storage section 1720 has a display pattern storage section 1722 , an image synthesis condition storage section 1724 , and a genuine product determination condition storage section 1726 .

In this embodiment, the management information storage unit 1720 stores various information used to control image output. The management information storage unit 1720 may store information generated in the anonymization process.

In this embodiment, the display pattern storage unit 1722 stores information (sometimes referred to as a display pattern) for controlling the timing at which at least one of the original image and the obscuration image is displayed. The display pattern storage section 1722 may store information for controlling the cycle of the display pattern. The display pattern is exemplified by the configuration of the output frame, the variation pattern of the output frame, and the like. For example, in order to prevent peeping by a third party 34, the constituent elements of each output frame may vary periodically between a plurality of consecutive output frames. For example, the frame period Pf increases or decreases according to a predetermined pattern, and the pattern is repeated for each predetermined period. In this case, the display pattern storage section 1722 may store information indicating the variation pattern of output frames in a frame group composed of a predetermined number of output frames.

In one embodiment, the display pattern storage unit 1722 stores display pattern identification information and information indicating the configuration of an output frame in association with each other. The configuration of the output frame includes the correspondence relationship between the input frame and the output frame, the number of subframes included in the output frame, the position of the subframe where the original image is displayed in the output frame, and the subframe where the occultation image is displayed in the output frame. Examples include the position of the frame, the frame period Pf of the output frame, the length Pso of the subframe in which the original image is displayed, and the length Psj of the subframe in which the occultation image is displayed.

In another embodiment, the display pattern storage unit 1722 may store the identification information of the display pattern and the information indicating the variation pattern of the output frame in association with each other. The information indicating the variation pattern of output frames includes information indicating the number of output frames constituting a frame group, information indicating the configuration of output frames that vary in the frame group, information indicating the increasing/decreasing pattern of the varying configuration, and the like. exemplified.

In this embodiment, the display pattern storage section 1722 may store information for controlling the timing of opening and closing the optical shutter 142 of the eyewear device 140 (sometimes referred to as an opening/closing pattern). The display pattern storage section 1722 may store a corresponding opening/closing pattern for each of one or more display patterns. The opening/closing pattern includes the opening/closing period Pg of the optical shutter 142, the length of the period Pgo during which the optical shutter 142 is open, the length of the period Pgc during which the optical shutter 142 is closed, and the period Pgo in the opening/closing period Pg of the optical shutter 142. The position, the position of the period Pgc in the opening/closing period Pg of the optical shutter 142, and combinations thereof are exemplified.

The display pattern storage section 1722 may store, for each of one or more display patterns, identification information of the display pattern and identification information of the corresponding opening/closing pattern in association with each other. As a result, when the display terminal 120 and the eyewear device 140 share a database that stores the identification information of the opening/closing pattern and the information indicating the opening/closing pattern in association with each other, the optical shutter control unit 865 By transmitting information indicating the identification information of the opening/closing pattern to the eyewear device 140, the operation of the optical shutter 142 can be controlled.

For example, the image output management unit 950 distributes the database or update information of the database to one or more eyewear devices 140 in an environment where the eyewear devices 150 do not exist in the vicinity of the eyewear devices 140. . This further reduces the risk of leakage of confidential information.

In this embodiment, the image synthesizing condition storage unit 1724 stores information indicating conditions for synthesizing images by the image synthesizing unit 1140 . The above conditions include a condition for determining whether correction is necessary in the image correction unit 1134, a condition for determining the type of correction in the image correction unit 1134, and a setting value related to correction in the image correction unit 1134. and a condition for determining an image combining method in the image combining unit 1140, and the like. The above conditions may be information indicating that an instruction from the user 20 should be followed, or information defining a decision criterion.

As described above, the types of correction performed by the image correction unit 1134 include correction for reducing the color gradation range in consideration of the display characteristics of the display 122, correction described with reference to FIG. 13, and the like. . The image synthesizing condition storage unit 1724 stores, for example, whether or not the display characteristics of the display 122 meet a predetermined condition as a criterion for determining whether correction for reducing the color gradation range is necessary. The information shown may be stored.

In this embodiment, the genuine product determination condition storage unit 1726 stores the genuine product determination conditions. As mentioned above, the conditions for judging genuine products include the condition that the device to be judged is registered in the whitelist, the condition that the device is not registered in the blacklist, and the specification of the device does not meet certain conditions. The condition that the conditions match, the condition that information about a specific item related to the state of the device is obtained, and the control signal transmitted from the device to the display terminal 120 includes information indicating a predetermined character string. and whether or not a general regular expression (for example, TrueFalse is calculated using a regular expression function for character strings that is standardly installed in the OS) is satisfied.

In this embodiment, the display timing determination unit 1732 determines the timing for displaying the original image on the display 122 of the display terminal 120 . The display timing determination unit 1732 determines the timing for displaying the occultation image on the display 122 of the display terminal 120 . The display timing determination unit 1732 may determine to display the original image and the obscuration image on the display 122 in a time division manner. The display timing determination unit 1732 may determine to display the original image and the obscuration image on the display 122 in a time division manner when the genuine product determination unit 1740 does not detect any suspicious device.

The display timing determination unit 1732 may determine the timing to stop displaying the original image. For example, when the genuine product determination unit 1740 detects a suspicious device during execution of anonymization processing, the display timing determination unit 1732 determines to stop displaying the original image.

The display timing determination unit 1732 may determine to change the display timing of the original image. For example, when the genuine product determination unit 1740 detects a suspicious device during execution of anonymization processing, the display timing determination unit 1732 determines to change the display timing of the original image.

In this embodiment, the opening/closing timing determination section 1734 determines the timing for opening/closing the optical shutter 142 of the eyewear type device 140 . The opening/closing timing determination section 1734 may determine the timing for opening and closing the optical shutter 142 based on the timing for displaying the original image determined by the display timing determination section 1732 .

In this embodiment, the genuine product determination unit 1740 determines whether or not one or more peripheral devices detected by the peripheral device management unit 862 include a suspicious device that does not meet the genuine product determination conditions. Suspicious devices can thereby be detected.

In this embodiment, the control signal output section 1750 outputs various control signals. Control signal output section 1750 may output a control signal for controlling the operation of each section of control section 860 or may output a control signal for controlling the operation of eyewear device 140 .

For example, when the display timing determination section 1732 determines the timing for displaying the original image, the control signal output section 1750 may output a control signal indicating the timing to the output image generation section 940 . The control signal output section 1750 may output a control signal indicating the timing to display the original image to the output image generation section 940 . The control signal output section 1750 may output a control signal indicating the timing for displaying the original image to the optical shutter control section 865 . The control signal output section 1750 may output a control signal indicating the timing to the optical shutter control section 865 at or near the timing at which the original image is displayed.

When the display timing determination section 1732 determines the timing for displaying the obscuration image, the control signal output section 1750 may output a control signal indicating the timing to the output image generation section 940 . The control signal output section 1750 may output a control signal indicating the timing to display the obscuration image to the output image generation section 940 . The control signal output section 1750 may output to the optical shutter control section 865 a control signal indicating the timing for displaying the obscuration image. The control signal output section 1750 may output a control signal indicating the timing to the optical shutter control section 865 at or near the timing at which the obscuration image is displayed.

When the display timing determining section 1732 determines to stop displaying the original image, the control signal outputting section 1750 may output a control signal indicating to stop displaying the original image to the output image generating section 940 . The control signal output section 1750 may output to the optical shutter control section 865 a control signal indicating to stop displaying the original image.

When the display timing determination unit 1732 determines to change the display timing of the original image, the control signal output unit 1750 outputs a control signal indicating to change the display timing of the original image to the output image generation unit 940. good. The control signal output section 1750 may output a control signal indicating the display timing of the changed original image to the output image generation section 940 . The control signal output section 1750 may output these control signals to the optical shutter control section 865 .

When the opening/closing timing determining section 1734 determines the opening/closing timing of the optical shutter 142 , the control signal output section 1750 may output a control signal indicating the opening/closing timing of the optical shutter 142 to the optical shutter control section 865 . The control signal output section 1750 may output to the optical shutter control section 865 a synchronization signal for synchronizing switching of images on the display 122 with opening and closing of the optical shutter 142 on the eyewear type device 140 .

When the genuine product determination unit 1740 detects a suspicious device, or when the genuine product determination unit 1740 detects a suspicious device during execution of anonymization processing, the control signal output unit 1750 sends the eyewear device 140 a A control signal for requesting wired connection or a control signal for requesting re-pairing to the eyewear device 140 may be output to the optical shutter control section 865 . Upon obtaining control signals from the image display control section 864 , the optical shutter control section 865 may transmit these control signals to the eyewear device 140 .

If the eyewear device 140 has an audio output function, the control signal for requesting wired connection and the control signal for requesting re-pairing cause the eyewear device 140 to output audio corresponding to each request. It may be a signal for The control signal for requesting wired connection and the control signal for requesting re-pairing may be signals for causing the eyewear device 140 to open and close the optical shutters in a special pattern corresponding to each request.

After the above control signal is transmitted to the eyewear device 140, the communication control unit 866 may cancel all pairings related to the eyewear device. In this case, the communication control section 866 may reject re-pairing of the eyewear device 150 registered in the blacklist or the eyewear device 150 not registered in the whitelist. After confirming that the eyewear device 140 is connected by wire or re-pairing of the eyewear device 140 registered in the whitelist is confirmed, the optical shutter control section 865 stops displaying the original image. A control signal indicating that the display timing of the original image is to be changed, or a control signal including information indicating the display timing of the original image after the change may be transmitted to the eyewear device 140 . This suppresses leakage of confidential information.

When the genuine product determination unit 1740 detects a suspicious device, or when the genuine product determination unit 1740 detects a suspicious device during execution of anonymization processing, the control signal output unit 1750 outputs the radio wave transmitted by the communication unit 840. A control signal may be output to the communication control section 866 to make the strength weaker than the predetermined strength. This suppresses leakage of confidential information.

The display timing determination unit 1732 may be an example of a display control unit. Genuine product determination unit 1740 may be an example of a determination unit.

An example of correspondence between an input frame and an output frame will be described with reference to FIGS. 18 and 19. FIG. An input frame may be an example of an input image signal. An output frame may be an example of an output image signal. 18 and 19 schematically show an example of information processing by the display timing determination unit 1732, for example.

According to the embodiment described in connection with FIG. 18, there is a one-to-one correspondence between input frames and output frames. In this embodiment, multiple input frames are input to the image display controller 864 , including consecutive input frames 1822 , 1824 , 1826 and 1828 .

For example, the display timing determination unit 1732 determines to generate the output frame 1842 corresponding to the input frame 1822 . In this embodiment, output frame 1842 includes subframe 1852 and subframe 1862 . The display terminal 120 processes the sub-frame 1852 to display the original image corresponding to the input frame 1822 in the desktop display area 212 of the display 122 . Sub-frame 1862 is processed by display terminal 120 to display the occult image corresponding to input frame 1822 in desktop display area 212 of display 122 .

Similarly, display timing determination section 1732 determines to generate output frame 1844 corresponding to input frame 1824 . In this embodiment, output frame 1844 includes subframe 1854 and subframe 1864 . The display timing determination unit 1732 determines to generate the output frame 1846 corresponding to the input frame 1826 . In this embodiment, output frame 1846 includes subframe 1856 and subframe 1866 . The display timing determination unit 1732 determines to generate an output frame 1848 corresponding to the input frame 1828 . In this embodiment, output frame 1848 includes subframe 1858 and subframe 1868 .

Subframe 1854 may be a frame for outputting the original image corresponding to input frame 1824 . Subframe 1864 may be a frame for outputting an occultation image corresponding to input frame 1824 . Subframe 1856 may be a frame for outputting the original image corresponding to input frame 1826 . Subframe 1866 may be a frame for outputting an occultation image corresponding to input frame 1826 . Subframe 1858 may be a frame for outputting the original image corresponding to input frame 1828 . Subframe 1868 may be a frame for outputting an occultation image corresponding to input frame 1828 .

According to the embodiment described in connection with FIG. 19, there is not a one-to-one correspondence between input frames and output frames, eg some of the input frames are discarded. In this embodiment, multiple input frames are input to the image display controller 864 , including consecutive input frames 1822 , 1824 , 1826 and 1828 .

For example, the display timing determination unit 1732 determines to generate the output frame 1942 corresponding to the input frame 1822 . In this embodiment, output frame 1942 includes subframe 1952 and subframe 1962 . The display terminal 120 processes the sub-frame 1952 to display the original image corresponding to the input frame 1822 in the desktop display area 212 of the display 122 . Sub-frame 1962 is processed by display terminal 120 to display an occult image corresponding to input frame 1822 in desktop display area 212 of display 122 .

According to the present embodiment, subframes 1952 and 1962 are set so that the total frame period of input frames 1822 and 1824 matches the total frame period of subframes 1952 and 1962 . Each frame period is determined. It is also decided to discard input frame 1824 .

Similarly, display timing determination section 1732 determines to generate output frame 1946 corresponding to input frame 1826 . In this embodiment, output frame 1946 includes subframe 1956 and subframe 1966 . Display terminal 120 processes subframe 1956 to display the original image corresponding to input frame 1826 in desktop display area 212 of display 122 . Sub-frame 1966 is processed by display terminal 120 to display the occult image corresponding to input frame 1826 in desktop display area 212 of display 122 .

According to this embodiment, subframe 1956 and subframe 1966 are set so that the total frame period of input frame 1826 and input frame 1828 matches the total frame period of subframe 1956 and subframe 1966 . Each frame period is determined. It is also decided to discard input frame 1828 .

For example, when the rate of change of the position, shape, and size of the target window is greater than a predetermined value during execution of the anonymization process, according to the embodiment described with reference to FIG. operation becomes unnatural, and the anonymization of the target window may become insufficient. In contrast, according to the embodiment described in connection with FIG. 19, the user 20 is more likely to be targeted as compared to the embodiment described in connection with FIG. Windows can be easily manipulated. Also, compared to the embodiment described in connection with FIG. 18, the window of interest is well masked.

If the rate of change of the target window's position, shape and size is greater than a predetermined value, the embodiment described with respect to FIG. 19 may be employed. On the other hand, if the rate of change of the target window's position, shape and size is less than the predetermined value, the embodiment described in connection with FIG. 18 may be employed. The embodiment described with reference to FIG. 18 and the embodiment described with reference to FIG. 19 may be switched depending on the change speed of the position, shape and size of the target window.

FIG. 20 schematically shows an example of the internal configuration of the eyewear type device 140. As shown in FIG. Note that the eyewear type device 150 may also have the same configuration as the eyewear type device 140 .

In this embodiment, the eyewear device 140 comprises an optical shutter 142 , a communication section 2020 , a control section 2030 and a storage section 2040 . In this embodiment, the control unit 2030 has a state management unit 2032 , an opening/closing control unit 2034 and a communication control unit 2036 . In this embodiment, the storage section 2040 has a device information storage section 2042 and an opening/closing pattern storage section 2044 .

In this embodiment, the communication unit 2020 transmits and receives information to and from equipment external to the eyewear type device 140 . The communication unit 2020 may transmit/receive information to/from an external device using a wired communication method, or may transmit/receive information to/from an external device using a wireless communication method. For example, the communication unit 2020 is configured to communicate with the display terminal 120 . The communication unit 2020 may be configured to be wirelessly communicable with the display terminal 120 .

In this embodiment, the controller 2030 controls the operation of the eyewear device 140 . The control unit 2030 may perform the recognition enabling process described above by controlling the operation of the optical shutter 142 .

In this embodiment, the state management section 2032 manages the state of each section of the eyewear device 140 . In one embodiment, the state manager 2032 manages identification information (sometimes referred to as a device ID) for the eyewear device 140 . The device ID may be identification information unique to the eyewear device 140 or identification information assigned to the user 20 or the eyewear device 140 by the information concealment system 100 or display terminal 120 . In another embodiment, the state manager 2032 manages the remaining battery power of the eyewear device 140 . The state management section 2032 acquires information indicating the remaining amount of the battery (not shown) arranged in the eyewear device 140 and stores the information in the device information storage section 2042 .

In this embodiment, the opening/closing control section 2034 controls various operations related to the optical shutter 142 . The opening/closing control section 2034 may control the opening/closing operation of the optical shutter 142 . For example, the opening/closing control section 2034 acquires the control signal described above from the image display control section 864 . The opening/closing control section 2034 controls the opening/closing operation of the optical shutter 142 based on the above control signal.

In one embodiment, the opening/closing control section 2034 may (i) control the opening/closing operation of the optical shutter 142 based on the synchronization signal included in the control signal output by the image display control section 864 . For example, the opening/closing control unit 2034 opens the optical shutter 142 when the synchronization signal indicates a first value (eg, 0), and opens the optical shutter 142 when the synchronization signal indicates a second value (eg, 1). The optical shutter 142 is closed.

In another embodiment, the opening/closing control unit 2034 controls (i) the display pattern included in the control signal output by the image display control unit 864 and (ii) the synchronization signal output by the image display control unit 864 or the eyewear The opening/closing operation of the optical shutter 142 may be controlled based on the output signal of the clock arranged in the mold device 140 . For example, when the opening/closing control unit 2034 receives a control signal including information indicating an image display pattern on the display 122 , the opening/closing control unit 2034 stores the information indicating the display pattern in the opening/closing pattern storage unit 2044 . The opening/closing control section 2034 determines the opening/closing pattern of the optical shutter 142 based on the image display pattern on the display 122 . The opening/closing control unit 2034 stores information indicating the opening/closing pattern of the optical shutter 142 in the opening/closing pattern storage unit 2044 .

Upon receiving the synchronization signal from the display terminal 120, the opening/closing control unit 2034 opens and closes the optical shutter 142 based on the synchronization signal so that the display pattern of the image on the display 122 and the opening/closing pattern of the optical shutter 142 are synchronized. control behavior. The opening/closing control unit 2034 may control the opening/closing operation of the optical shutter 142 based on the above output signal of the clock so that the display pattern of the image on the display 122 and the opening/closing pattern of the optical shutter 142 are synchronized. .

In another embodiment, the opening/closing control unit 2034 controls (i) the opening/closing pattern included in the control signal output by the image display control unit 864, and (ii) the synchronization signal output by the image display control unit 864, or the eyewear control unit 864. The opening/closing operation of the optical shutter 142 may be controlled based on the output signal of the clock arranged in the mold device 140 . For example, upon receiving a control signal including information indicating the opening/closing pattern of the optical shutter 142 , the opening/closing control section 2034 stores the information indicating the opening/closing pattern in the opening/closing pattern storage section 2044 . The opening/closing control unit 2034 stores the identification information of the opening/closing pattern described with reference to FIG. You can get database update information. The opening/closing control unit 2034 may store the database or update information of the database in the opening/closing pattern storage unit 2044 .

Upon receiving the synchronization signal from the display terminal 120, the opening/closing control unit 2034 opens and closes the optical shutter 142 based on the synchronization signal so that the display pattern of the image on the display 122 and the opening/closing pattern of the optical shutter 142 are synchronized. control behavior. The opening/closing control unit 2034 may control the opening/closing operation of the optical shutter 142 based on the above output signal of the clock so that the display pattern of the image on the display 122 and the opening/closing pattern of the optical shutter 142 are synchronized. .

In this embodiment, the communication control section 2036 controls the communication section 2020 . Communication between each section of the control section 2030 and equipment outside the eyewear type device 140 may be controlled. The communication control section 2036 may output a radio signal including the identification information of the eyewear type device 140 from the communication section 2020 according to the radio communication method. Examples of the wireless signal include a probe signal including an ID unique to a device such as a MAC address or Bluetooth (registered trademark) Device ID (sometimes referred to as UUID).

In this embodiment, the storage unit 2040 stores various information. As described above, the device information storage unit 2042 stores, for example, identification information of the eyewear device 140 and information indicating the status of each unit of the eyewear device 140 . The opening/closing pattern storage unit 2044 stores, for example, information indicating an image display pattern on the display 122, information indicating an opening/closing pattern of the optical shutter 142, and the like.

FIG. 21 illustrates an example computer 3000 upon which aspects of the present invention may be embodied, in whole or in part. At least part of the information concealment system 100 may be realized by the computer 3000 . For example, the display terminal 120 or part thereof is implemented by the computer 3000 . For example, eyewear device 140 or a portion thereof is implemented by computer 3000 . For example, eyewear device 150 or a portion thereof is implemented by computer 3000 .

Programs installed on the computer 3000 cause the computer 3000 to function as one or more "parts" of operations or one or more "parts" of an apparatus according to embodiments of the invention, or to and/or cause the computer 3000 to perform processes or steps of processes according to embodiments of the present invention. Such programs may be executed by CPU 3012 to cause computer 3000 to perform certain operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

Computer 3000 according to this embodiment includes CPU 3012 , RAM 3014 , GPU 3016 , and display device 3018 , which are interconnected by host controller 3010 . Computer 3000 also includes input/output units such as communication interface 3022 , hard disk drive 3024 , DVD-ROM drive 3026 and IC card drive, which are connected to host controller 3010 via input/output controller 3020 . The computer also includes legacy input/output units such as ROM 3030 and keyboard 3042 , which are connected to input/output controller 3020 via input/output chip 3040 .

The CPU 3012 operates according to programs stored in the ROM 3030 and RAM 3014, thereby controlling each unit. The GPU 3016 retrieves image data generated by the CPU 3012 into itself, such as a frame buffer provided in RAM 3014 , and causes the image data to be displayed on the display device 3018 .

Communication interface 3022 communicates with other electronic devices over a network. Hard disk drive 3024 stores programs and data used by CPU 3012 within computer 3000 . DVD-ROM drive 3026 reads programs or data from DVD-ROM 3001 and provides programs or data to hard disk drive 3024 via RAM 3014 . The IC card drive reads programs and data from IC cards and/or writes programs and data to IC cards.

ROM 3030 stores therein programs that are dependent on the hardware of computer 3000, such as a boot program that is executed by computer 3000 upon activation. Input/output chip 3040 may also connect various input/output units to input/output controller 3020 via parallel ports, serial ports, keyboard ports, mouse ports, and the like.

A program is provided by a computer-readable storage medium such as a DVD-ROM 3001 or an IC card. The program is read from a computer-readable storage medium, installed in hard disk drive 3024 , RAM 3014 , or ROM 3030 , which are also examples of computer-readable storage media, and executed by CPU 3012 . The information processing described within these programs is read by computer 3000 to provide coordination between the programs and the various types of hardware resources described above. An apparatus or method may be configured by implementing information operations or processing according to the use of computer 3000 .

For example, when communication is performed between the computer 3000 and an external device, the CPU 3012 executes a communication program loaded into the RAM 3014 and sends communication processing to the communication interface 3022 based on the processing described in the communication program. you can command. Under the control of the CPU 3012, the communication interface 3022 reads the transmission data stored in the transmission buffer area provided in the recording medium such as the RAM 3014, the hard disk drive 3024, the DVD-ROM 3001, or the IC card. Data is transmitted to the network, or received data received from the network is written in a receive buffer area or the like provided on the recording medium.

In addition, the CPU 3012 causes the RAM 3014 to read all or necessary portions of files or databases stored in external recording media such as a hard disk drive 3024, a DVD-ROM drive 3026 (DVD-ROM 3001), an IC card, etc. Various types of processing may be performed on the data in RAM 3014 . CPU 3012 may then write back the processed data to an external recording medium.

Various types of information, such as various types of programs, data, tables, and databases, may be stored on recording media and subjected to information processing. CPU 3012 performs various types of operations on data read from RAM 3014, information processing, conditional decisions, conditional branching, unconditional branching, and information retrieval as specified throughout this disclosure and by instruction sequences of programs. Various types of processing may be performed, including /replace, etc., and the results written back to RAM 3014 . Also, the CPU 3012 may search for information in a file in a recording medium, a database, or the like. For example, when a plurality of entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in the recording medium, the CPU 3012 selects the first attribute from among the plurality of entries. search for an entry that matches the specified condition of the attribute value of the attribute, read the attribute value of the second attribute stored in the entry, and thereby determine the first attribute that satisfies the predetermined condition An attribute value of the associated second attribute may be obtained.

The programs or software modules described above may be stored in a computer readable storage medium on or near computer 3000 . In addition, a recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, whereby the above program can be transferred via a network. provided to the computer 3000;

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is obvious to those skilled in the art that various modifications and improvements can be made to the above embodiments. In addition, matters described with respect to a specific embodiment can be applied to other embodiments as long as they are not technically inconsistent. It is clear from the description of the scope of claims that forms with such modifications or improvements can also be included in the technical scope of the present invention.

The execution order of each process such as actions, procedures, steps, and stages in the devices, systems, programs, and methods shown in the claims, the specification, and the drawings is particularly "before", "before etc., and it should be noted that they can be implemented in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the specification, and the drawings, even if the description is made using "first," "next," etc. for the sake of convenience, it means that it is essential to carry out in this order. not a thing

This specification discloses, for example, the following matters. It should be noted that each step of the control method shown below may be executed by a computer.
[Item A-1]
A control device for controlling a display device equipped with a display device for displaying an image,
a display control unit that controls the display device;
a peripheral device detection unit having at least one of an optical shutter and an imaging device and detecting one or more peripheral devices existing around the display device;
Whether the one or more peripheral devices detected by the peripheral device detection unit include a suspicious device that is a device that does not meet a genuine product determination condition that is a condition related to devices for which confidential information recognition enablement processing is permitted A determination unit that determines whether or not
with
The display control unit is
(a) when the determination unit does not determine that the suspicious device is included in the one or more peripheral devices;
A configuration capable of executing a first anonymization process for displaying a confidential image for displaying the confidential information and an obstructive image for obstructing visual recognition of at least a part of the confidential image on the display device in a time division manner. has been
(b) when the determination unit determines that the suspicious device is included in the one or more peripheral devices;
a second anonymization process for anonymizing at least part of the confidential information in a manner different from the first anonymization process;
The second anonymization process is
processing for stopping display of the confidential image on the display device;
processing for changing the timing at which the confidential image is displayed on the display device;
Processing for shrinking the window in which the confidential information is displayed,
Processing for moving the above window to the back of other windows,
Processing for changing the display position of the window, and
Processing for changing the color tone, contrast or brightness of the confidential image,
including at least one of
Control device.
[Item A-2]
The processing for changing the timing at which the confidential image is displayed on the display device includes:
processing for changing the pattern for defining display and non-display of the confidential image;
A process for changing the start time of the pattern, and
processing for changing the cycle of the pattern;
is at least one of
The control device according to item A-1.
[Item A-3]
The above disturbing image is
An image for obstructing the visibility of the image displayed on substantially the entire screen display area of the display device among the confidential images,
An image for obstructing visibility of an image displayed substantially in the center of the screen display area of the display device among the confidential images;
An image for obstructing visibility of an image displayed on a specific scanning line of the display device among the confidential images; and
An image for obstructing the visibility of an image having a specific color tone or brightness among the confidential images,
including at least one of
A control device according to item A-1 or item A-2.
[Item A-4]
further comprising a transmission control unit that controls a transmitter configured to communicate with at least one of the one or more peripheral devices;
The transmission control unit is
When the display control unit determines to change the timing at which the confidential image is displayed,
Changing the timing at which the confidential image is displayed for an authorized device, which has an optical shutter and satisfies the genuine product determination condition, among the one or more peripheral devices detected by the peripheral device detection unit. controlling the transmitter such that information indicative of
The control device according to any one of items A-1 to A-3.
[Item A-5]
The transmitter is configured to be capable of wireless communication with the authorized device,
The transmission control unit is
When the display control unit determines to change the timing at which the confidential image is displayed,
controlling the transmitter so that the radio field strength of a radio wave that transmits information indicating a change in the timing at which the confidential image is displayed is weaker than a predetermined strength;
A control device according to item A-4.
[Item A-6]
the image displayed on the display device includes one or more display windows for displaying information to be presented to a user;
The confidential image includes at least an image of an image display area of a display window for displaying the confidential information.
The control device according to any one of items A-1 to A-5.
[Item A-7]
The display control unit is
A full-screen window covering substantially the entire screen display area of the display device is arranged in front of the one or more display windows,
so that the confidential image and the disturbing image for each frame are displayed in the full screen window in a time division manner,
controlling the display device;
A control device according to item A-6.
[Item A-8]
a secret image generation unit that generates the secret image;
a disturbing image generation unit that generates the disturbing image;
a target identifying unit that identifies a target window to be subjected to the anonymization process among the one or more display windows;
a window information acquiring unit for acquiring window information indicating the position, shape and size of the target window specified by the target specifying unit;
further comprising
The interference image generation unit is configured to determine the position, shape, and size at which the interference image is displayed on the display device based on the position, shape, and size of the target window indicated by the window information acquired by the window information acquisition unit. determine the size of
A control device according to item A-6 or item A-7.
[Item A-9]
The disturbance image generation unit generates a disturbance window for displaying the disturbance image,
The display control unit (i) arranges the target window in front of the disturbance window when the confidential image is displayed on the display device, and (ii) displays the disturbance image on the display device. If so, control the display device so that the disturbing window is placed in front of the target window;
A controller according to item A-8.
[Item A-10]
A program for causing a computer to function as the control device according to any one of items A-1 to A-9.
[Item A-11]
a display device for displaying an image;
a control device according to any one of items A-1 to A-9, which controls the display device;
display device.
[Item A-12]
A control method for controlling a display device having a display device for displaying an image, comprising:
a display control stage for controlling the display device;
a peripheral device detection step having at least one of an optical shutter and an imaging device and detecting one or more peripheral devices existing around the display device;
The one or more peripheral devices detected in the peripheral device detection step include a suspicious device that is a device that does not meet the genuine product determination condition, which is a condition regarding the device for which confidential information recognition enablement processing is permitted. a determination step of determining whether or not
has
The display control stage is
(a) when it is determined in the determination step that the suspicious device is not included in the one or more peripheral devices, a confidential image for displaying confidential information and at least part of the confidential image are visually recognized; a step of determining to execute a first anonymization process of causing the display device to time-divisionally display an obstruction image for obstructing the
(b) when it is determined in the determination step that the suspicious device is included in the one or more peripheral devices, anonymizing at least part of the confidential information in a manner different from the first anonymizing process; a step of deciding to perform a second anonymization process to
including
The second anonymization process is
processing for stopping display of the confidential image on the display device;
processing for changing the timing at which the confidential image is displayed on the display device;
Processing for shrinking the window in which the confidential information is displayed,
Processing for moving the above window to the back of other windows,
Processing for changing the display position of the window, and
Processing for changing the color tone or brightness of the confidential image,
including at least one of
control method.

This specification discloses, for example, the following matters. It should be noted that each step of the control method shown below may be executed by a computer.
[Item B-1]
A control device for controlling a display device equipped with a display device for displaying an image,
a confidential image generation unit that generates a confidential image for displaying confidential information, which is information to be processed for confidentiality;
an obstructing image generation unit that generates an obstructing image for obstructing viewing of at least part of the confidential image;
a display control unit that controls the display device so that the display device displays the confidential image and the disturbing image in a time division manner for each frame;
A controller.
[Item B-2]
further comprising a hidden area determination unit that determines a hidden area, which is an area in the hidden image that should be obstructed from being viewed by a third party, based on a change in color tone of the hidden image;
The obstruction image generation unit generates the obstruction image for obstructing the visibility of the hidden area.
The control device according to item B-1.
[Item B-3]
The hidden area determination unit is
a position specifying unit that specifies a position of the secret information in the secret image based on a change in color tone of a plurality of pixels forming the secret image;
a range determining unit that determines a range of each of the one or more hidden areas so that at least part of the confidential information is covered by the one or more hidden areas;
having
A control device according to item B-2.
[Item B-4]
The interference image generation unit is
(i) converting the color tone of at least some of the one or more pixels arranged inside the concealment area; (ii) brightness or brightness of at least some of the one or more pixels arranged outside the concealment area; By adjusting the
generating the sabotage image corresponding to the confidential image;
The control device according to item B-2 or item B-3.
[Item B-5]
the image displayed on the display device includes one or more display windows for displaying information to be presented to a user;
The confidential image includes at least an image of an image display area of a display window for displaying the confidential information.
The control device according to item B-1.
[Item B-6]
a target identifying unit that identifies a target window to be subjected to the anonymization process among the one or more display windows;
a window information acquiring unit for acquiring window information indicating the position, shape and size of the target window specified by the target specifying unit;
further comprising
The interference image generation unit is configured to determine the position, shape, and size at which the interference image is displayed on the display device based on the position, shape, and size of the target window indicated by the window information acquired by the window information acquisition unit. determine the size of
A control device according to item B-5.
[Item B-7]
a hidden area determination unit that determines a hidden area, which is an area where viewing by a third party is to be obstructed, in the image in the image display area, based on a change in color tone of the image in the image display area of the display window; ,
The obstruction image generation unit generates the obstruction image for obstructing the visibility of the hidden area.
the control device according to item B-6;
[Item B-8]
The display control unit is
A full-screen window covering substantially the entire screen display area of the display device is arranged in front of the one or more display windows,
so that the confidential image and the disturbing image for each frame are displayed in the full screen window in a time division manner,
controlling the display device;
The control device according to any one of items B-5 to B-7.
[Item B-9]
The display control unit is
(a) generating a full-screen window that covers substantially the entire screen display area of the display device;
(b) (i) When the confidential image is displayed on the display device, the full screen window having a transparent area or a transparent area at least in the area where the confidential image is displayed is the one or more display windows. and (ii) when the disturbing image is displayed on the display device, the full-screen window on which the disturbing image is drawn is arranged in front of the one or more display windows. to control the display device,
The control device according to any one of items B-5 to B-7.
[Item B-10]
a pointer detection unit that detects the position of the pointer in the screen display area of the display device;
a color information acquisition unit that acquires information about a transparent color or a color treated as a transparent color;
further comprising
The interference image generation unit is
(i) generating the disturbing image in which the color of the pointer region, which is the region containing the indicated position of the pointer, is the transparent color or a color treated as the transparent color;
The control device according to item B-1.
[Item B-11]
a display device for displaying an image;
a control device according to any one of items B-1 to B-10, which controls the display device;
display device.
[Item B-12]
A program for causing a computer to function as the control device according to any one of items B-1 to B-10.
[Item B-13]
A control method for controlling a display device having a display device for displaying an image, comprising:
a secret image generating stage for generating a secret image for displaying secret information, which is information to be subjected to anonymization processing;
a disturbing image generation step of generating an disturbing image for disturbing the viewing of the confidential image;
a display control step of controlling the display device so that the display device displays the confidential image and the disturbing image in a time division manner for each frame;
A control method.

This specification discloses, for example, the following matters. It should be noted that each step of the control method shown below may be executed by a computer.
[Item C-1]
A control device for controlling a display device comprising a display device for displaying an image including one or more display windows for displaying information presented to a user,
a confidential image generation unit that generates a confidential image for displaying confidential information, which is information to be processed for confidentiality;
an obstructing image generation unit that generates an obstructing image for obstructing viewing of the confidential image;
a display control unit that controls the display device so that the display device displays the confidential image and the disturbing image in a time division manner for each frame;
with
The confidential image includes at least an image of an image display area of a display window for displaying the confidential information.
Control device.
[Item C-2]
a target identifying unit that identifies a target window to be subjected to the anonymization process among the one or more display windows;
a window information acquiring unit for acquiring window information indicating the position, shape and size of the target window specified by the target specifying unit;
further comprising
The interference image generation unit is configured to determine the position, shape, and size at which the interference image is displayed on the display device based on the position, shape, and size of the target window indicated by the window information acquired by the window information acquisition unit. determine the size of
The control device according to item C-1.
[Item C-3]
The disturbance image generation unit is configured to, in a screen display area of the display device, set a disturbance region in which the disturbance image is displayed when the disturbance image is displayed on the display device, and a disturbance region in which the confidential image is displayed on the display device. determining the position, shape and size of the disturbing image in the screen display area so as to cover the display area where the confidential image is displayed when
A control device according to item C-2.
[Item C-4]
further comprising an instruction receiving unit that receives a change instruction for changing at least one of the position, shape and size of the target window;
The disturbance image generator determines the position, shape and size of the disturbance image according to the speed at which at least one of the position, shape and size of the target window is changed.
A control device according to item C-3.
[Item C-5]
The display control unit is
A full-screen window covering substantially the entire screen display area of the display device is arranged in front of the one or more display windows,
so that the confidential image and the disturbing image for each frame are displayed in the full screen window in a time division manner,
controlling the display device;
The control device according to any one of items C-1 to C-4.
[Item C-6]
The display control unit is
(a) generating the full screen window;
(b) (i) when the confidential image is displayed on the display device, the full-screen window in which the confidential image is drawn is arranged in front of the one or more display windows; and (ii) When the disturbing image is displayed on the display device, the display device is controlled so that the full-screen window on which the disturbing image is drawn is arranged in front of the one or more display windows. ,
A control device according to item C-5.
[Item C-7]
The display control unit is
(a) generating the full screen window;
(b) (i) When the confidential image is displayed on the display device, the full screen window having a transparent area or a transparent area at least in the area where the confidential image is displayed is the one or more display windows. and (ii) when the disturbing image is displayed on the display device, the full-screen window on which the disturbing image is drawn is arranged in front of the one or more display windows. to control the display device,
A control device according to item C-5.
[Item C-8]
The disturbance image generation unit generates a disturbance window for displaying the disturbance image,
The display control unit (i) arranges the target window in front of the disturbance window when the confidential image is displayed on the display device, and (ii) displays the disturbance image on the display device. If so, control the display device so that the disturbing window is placed in front of the target window;
The control device according to any one of items C-2 to C-4.
[Item C-9]
The disturbance image generation unit generates a complementary color image having a complementary color relationship with the confidential image as the disturbance image.
The control device according to any one of items C-1 to C-8.
[Item C-10]
The interference image generation unit generates the complementary color image in consideration of the display characteristics of the display device.
A controller according to item C-8.
[Item C-11]
The secret image generation unit generates a first secret image and a second secret image for displaying the secret image when displayed on the display device at a display speed of 15 frames per second or more,
The disturbance image generation unit generates a first disturbance image and a second disturbance image for displaying the disturbance image when switched at a display speed of 15 frames per second or more and displayed on the display device,
The first secret image and the second secret image are different in color tone or brightness of corresponding regions,
The first disturbing image and the second disturbing image differ in color tone or brightness of corresponding regions.
A control device according to item C-8 or item C-9.
[Item C-12]
a display device for displaying an image;
a control device according to any one of items C-1 to C-11, which controls the display device;
display device.
[Item C-13]
A program for causing a computer to function as the control device according to any one of items C-1 to C-11.
[Item C-14]
A control method for controlling a display device comprising a display device for displaying an image including one or more display windows for displaying information presented to a user, comprising:
a secret image generating stage for generating a secret image for displaying secret information, which is information to be subjected to anonymization processing;
a disturbing image generation step of generating an disturbing image for disturbing the viewing of the confidential image;
a display control step of controlling the display device so that the display device displays the confidential image and the disturbing image in a time division manner for each frame;
has
The confidential image includes at least an image of an image display area of a display window for displaying the confidential information.
control method.

20 User 32 Third Party 34 Third Party 100 Information Concealment System 120 Display Terminal 122 Display 140 Eyewear Type Device 142 Optical Shutter 150 Eyewear Type Device 152 Optical Shutter 174 Authentic Product Judging Section 200 Desktop Image 212 Desktop Display Area 214 Taskbar Display area 220 active window 222 title area 224 work area 226 cursor 240 inactive window 242 title area 244 work area 260 pointer 302 output frame 312 subframe 314 subframe 320 desktop image 322 window image 324 window image 326 pointer image 340 desktop image 342 window Image 344 Window image 346 Pointer image 500 Desktop image 524 Fusion image 602 Output frame 612 Subframe 614 Subframe 616 Subframe 618 Subframe 660 Desktop image 662 Dummy image 680 Desktop image 682 Dummy image 700 Desktop image 724 Fusion image 820 Input section 840 Communication unit 860 Control unit 862 Peripheral device management unit 863 Application execution unit 864 Image display control unit 865 Optical shutter control unit 866 Communication control unit 922 User input reception unit 924 Pointer position determination unit 926 Window management unit 932 Input image generation unit 934 Input image Buffer 936 Complementary color image generator 938 Complementary color image buffer 940 Output image generator 950 Image output manager 1000 Data table 1022 Window ID
1024 Information 1032 Process ID
1034 information 1042 information 1044 information 1052 information 1122 display window generation unit 1124 obscuration window generation unit 1132 superimposition state determination unit 1134 image correction unit 1140 image composition unit 1200 image 1210 background layer 1220 window layer 1222 display window 1224 output layer 1230 obscuration window frame 1322 subframe 1324 subframe 1326 subframe 1328 subframe 1342 desktop image 1344 desktop image 1346 desktop image 1348 desktop image 1362 area 1364 area 1366 area 1368 area 1410 background image 1420 desktop image 1440 desktop image 1510 background image 1520 desktop image 1540 desktop image Image 1620 Desktop image 1640 Desktop image 1720 Management information storage unit 1722 Display pattern storage unit 1724 Image synthesis condition storage unit 1726 Genuine product determination condition storage unit 1732 Display timing determination unit 1734 Open/close timing determination unit 1740 Genuine product determination unit 1750 Control signal output unit 1822 input frame 1824 input frame 1826 input frame 1828 input frame 1842 output frame 1844 output frame 1846 output frame 1848 output frame 1852 subframe 1854 subframe 1856 subframe 1858 subframe 1862 subframe 1864 subframe 1866 subframe 1868 subframe 1942 output Frame 1946 Output frame 1952 Subframe 1956 Subframe 1962 Subframe 1966 Subframe 2020 Communication unit 2030 Control unit 2032 State management unit 2034 Opening/closing control unit 2036 Communication control unit 2040 Storage unit 2042 Device information storage unit 2044 Opening/closing pattern storage unit 3000 Computer 3001 DVD-ROM
3010 host controller 3012 CPU
3014 RAM
3016 GPUs
3018 display device 3020 input/output controller 3022 communication interface 3024 hard disk drive 3026 DVD-ROM drive 3030 ROM
3040 input/output chip 3042 keyboard

Claims (12)

A control device for controlling a display device equipped with a display device for displaying an image,
a display control unit that controls the display device;
a peripheral device detection unit having at least one of an optical shutter and an imaging device and detecting one or more peripheral devices existing around the display device;
Whether the one or more peripheral devices detected by the peripheral device detection unit include a suspicious device that is a device that does not meet a genuine product determination condition that is a condition related to devices for which confidential information recognition enablement processing is permitted A determination unit that determines whether or not
with
The display control unit
(a) when the determination unit does not determine that the suspicious device is included in the one or more peripheral devices;
A configuration capable of executing a first anonymization process for displaying a confidential image for displaying the confidential information and an obstructive image for obstructing visual recognition of at least a part of the confidential image on the display device in a time division manner. has been
(b) when the determination unit determines that the suspicious device is included in the one or more peripheral devices;
a second anonymization process for anonymizing at least part of the confidential information in a manner different from the first anonymization process,
The second anonymization process includes:
processing for stopping display of the confidential image on the display device;
processing for changing the timing at which the confidential image is displayed on the display device;
processing for shrinking the window in which the confidential information is displayed;
a process for moving the window behind other windows;
a process for changing the display position of the window; and
processing for changing the color tone, contrast or brightness of the confidential image;
including at least one of
Control device. The processing for changing the timing at which the confidential image is displayed on the display device includes:
processing for changing a pattern for defining display and non-display of the confidential image;
a process for changing the beginning of the pattern; and
processing for changing the period of the pattern;
is at least one of
A control device according to claim 1 . The interfering image is
An image for obstructing the visibility of the image displayed on substantially the entire screen display area of the display device among the confidential images,
An image for obstructing visibility of an image displayed substantially in the center of the screen display area of the display device among the confidential images;
An image for obstructing visibility of an image displayed on a specific scanning line of the display device among the confidential images; and
An image for obstructing the visual recognition of an image having a specific color tone or brightness among the confidential images,
including at least one of
The control device according to claim 1 or 2. further comprising a transmission control unit that controls a transmitter configured to communicate with at least one of the one or more peripheral devices;
The transmission control unit
When the display control unit determines to change the timing at which the confidential image is displayed,
Changing the timing at which the confidential image is displayed for an authorized device, which has an optical shutter and meets the genuine product determination condition, among the one or more peripheral devices detected by the peripheral device detection unit. controlling the transmitter such that information indicative of
Control device according to any one of claims 1 to 3. The transmitter is configured to be capable of wireless communication with the authorized device,
The transmission control unit
When the display control unit determines to change the timing at which the confidential image is displayed,
controlling the transmitter so that the radio wave intensity of a radio wave that transmits information indicating a change in the timing at which the confidential image is displayed is weaker than a predetermined intensity;
5. A control device according to claim 4. the image displayed on the display device includes one or more display windows for displaying information to be presented to a user;
The confidential image includes at least an image of an image display area of a display window for displaying the confidential information.
Control device according to any one of claims 1 to 5. The display control unit
a full-screen window that covers substantially the entire screen display area of the display device is arranged in front of the one or more display windows;
so that the confidential image and the disturbing image for each frame are displayed in the full screen window in a time division manner,
controlling the display device;
7. A control device according to claim 6. a secret image generation unit that generates the secret image;
a disturbing image generator that generates the disturbing image;
a target identifying unit that identifies a target window to be subjected to the anonymization process among the one or more display windows;
a window information acquiring unit for acquiring window information indicating the position, shape and size of the target window specified by the target specifying unit;
further comprising
The interference image generation unit is configured to determine the position, shape, and size at which the interference image is displayed on the display device, based on the position, shape, and size of the target window indicated by the window information acquired by the window information acquisition unit. determine the size of
The control device according to claim 6 or 7. The disturbance image generation unit generates a disturbance window for displaying the disturbance image,
(i) when the confidential image is displayed on the display device, the target window is arranged in front of the disturbance window; and (ii) the disturbance image is displayed on the display device. If so, control the display device so that the disturbing window is placed in front of the target window;
A control device according to claim 8 . A program for causing a computer to function as the control device according to any one of claims 1 to 9. a display device for displaying an image;
A display device comprising: the control device according to any one of claims 1 to 9, which controls the display device. A control method for controlling a display device having a display device for displaying an image, comprising:
a display control step for controlling the display device;
a peripheral device detection step having at least one of an optical shutter and an imaging device and detecting one or more peripheral devices existing around the display device;
The one or more peripheral devices detected in the peripheral device detection step include a suspicious device that is a device that does not meet a genuine product determination condition, which is a condition regarding devices for which confidential information recognition enablement processing is permitted. a determination step of determining whether or not
has
The display control step includes:
(a) when it is determined in the determination step that the suspicious device is not included in the one or more peripheral devices, a confidential image for displaying confidential information and at least part of the confidential image are visually recognized; a step of determining to execute a first anonymization process of causing the display device to display, in a time-division manner, an obstruction image for obstructing the
(b) when it is determined in the determination step that the suspicious device is included in the one or more peripheral devices, at least part of the confidential information is anonymized in a manner different from the first anonymization process; a step of deciding to perform a second anonymization process to
including
The second anonymization process includes:
processing for stopping display of the confidential image on the display device;
processing for changing the timing at which the confidential image is displayed on the display device processing for reducing the window in which the confidential information is displayed;
a process for moving the window behind other windows;
a process for changing the display position of the window; and
processing for changing the color tone or brightness of the confidential image;
including at least one of
control method.

Images