JP2011044094A - Display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a display device for correctly detecting an input operation by correctly removing a foreign matter on a display screen. <P>SOLUTION: The display device 1 includes: a display part 3; an imaging part 5; a defective area determining part 6; and an image processing part 7. The imaging part 5 receives light to be made incident from the display screen 10 having a touch panel function, and generates image data, based on the received light. The defective area determining part 6 detects the position of an object to be detected, which exists on the display screen 10, based on the image data generated by the imaging part 5. The image processing part 7 generates a defective area notification image including a defective area image, on the display screen 10, which expresses the position of the object to be detected detected by the defective area determining part 6, and also a non-defective area image, which expresses the position of an object other than the object to be detected on the display screen 10 and is different from the defective area image concerning a display form. The display part 3 displays the defective area notification image on the display screen 10. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a display device, and more particularly to a display device including a display screen having a touch panel function for receiving an input operation performed on the screen.

  Conventionally, touch panel technology has been developed in which a user performs an input operation by directly touching a display screen. In the touch panel technology, there are an optical method, a capacitance method, a resistive film method, an ultrasonic method, and the like as a method for detecting a user's input operation. Since these touch panel technologies are based on the premise that the user touches the screen directly, the problem that foreign matter such as dust, grease, dust, and scratches adheres to the screen is unavoidable. In particular, in an optical touch panel that detects an input operation from an image obtained by imaging a display screen, when a foreign object adheres, an inappropriate area appears in the captured image, and accurate detection cannot be performed. Arise.

  In order to solve this problem, there is a method of performing image processing for removing an inappropriate area from a captured image. As such an image processing method, Patent Document 1 describes an image processing method for acquiring a position of a foreign substance in a captured image and correcting the image at the acquired position to remove the foreign substance from the image. ing. In the image processing method of Patent Document 1, an image of a foreign object is displayed on a display screen, and the position of the foreign object is detected by the user specifying the position where the foreign object exists based on the image. Then, the foreign matter is removed from the image by replacing the pixel values of the pixels constituting the detected foreign matter with the pixel values of the neighboring pixels.

JP 2004-326265 A (November 18, 2004)

  However, in the image processing method described in Patent Document 1, it is necessary for the user to specify the position of the foreign matter, which is not convenient for the user. In addition, the image processing that replaces the pixel values of the pixels that make up the foreign object and the pixel values of the neighboring pixels is a process that estimates and replaces the image at the position where the foreign material exists from the neighboring images. There is a problem in that it cannot be replaced properly and accurate foreign matter removal cannot be achieved.

  The present invention has been made in view of the above problems, and an object thereof is to provide a display device capable of accurately removing foreign matter on a display screen and accurately detecting an input operation.

  In order to solve the above problems, the present invention is a display device having a display screen having a function of detecting an input operation input to the display screen, and receives light incident from the display screen. Imaging means for generating image data based on the light, detection means for detecting the position of the detected object present on the display screen based on the image data generated by the imaging means, and the display screen A defect area image indicating the position of the detected object detected by the detecting means and a position other than the detected object on the display screen, and a display format different from the defect area image. It is characterized by comprising image generating means for generating a defective area notification image including a defective area image, and display means for displaying the defective area notification image on the display screen.

  According to said structure, the display apparatus which concerns on this invention has a touch-panel function which detects the input operation which the user input into the display screen. Then, the display device according to the present invention detects a foreign object existing on the display screen as a detected part and notifies the user of the position of the foreign object, thereby prompting the user to remove the foreign object. That is, the imaging unit generates image data obtained by imaging a foreign object (detected portion) attached to the display screen based on light incident from the display screen. Then, the detection means detects the position of the detected object based on the image data generated by the imaging means. The image generation means represents a defect area image representing a position where the detected object exists on the display screen and a position other than the position where the detected object exists on the display screen based on the detection result of the detecting means. A defective area notification image including a non-defective area image is generated. At this time, the display format is different between the defect area image and the non-defect area image. And a display means displays the said defect area notification image on a display screen, and notifies a user of presence of a foreign material.

  In this way, from the image data obtained by imaging the display screen, the presence position of the foreign object is detected, an image for notifying the user of the presence position of the foreign object is generated, and the display screen that is the target of the imaging is displayed. By displaying, it is possible to perform input processing by imaging and output processing of the generated notification image on the same display screen. Therefore, it is possible to accurately notify the user of the position of the foreign matter on the display screen, and the foreign matter on the display screen can be reliably removed by removing the foreign matter at the position notified by the user. As a result, it is possible to prevent the detection accuracy from being reduced by the touch panel function of the display screen, and to accurately detect the input operation input to the display screen.

  In the display device according to the present invention, it is preferable that the image generation unit generates the defect area notification image so that the defect area image is displayed at a position of the detected object on the display screen. .

  As a result, the image generation unit generates the defect area notification image so that the defect area image is displayed at the position where the foreign object exists on the display screen, so that the user can more easily recognize the position of the foreign object on the display screen. can do.

  Further, in the display device according to the present invention, the image generation means includes a small screen image obtained by reducing the display screen, and in the small screen, a position corresponding to the position of the detected object on the display screen. Preferably, the defect area notification image is generated so that the defect area image is displayed.

  According to the above configuration, when the defect area notification image is displayed on the full screen, the existing display such as an input button for receiving a user operation input overlaps with the defect area image, and it is difficult for the user to confirm the position where the foreign object exists. Even so, by displaying the defect area notification image in the sub-screen displayed at a position that does not overlap with such existing display, the user can more easily recognize the position of the foreign matter on the display screen. . In other words, when the defect area notification image is displayed on the full screen, the defect area notification image is generated so that the defect area image is displayed at the position of the child screen corresponding to the position where the foreign object exists. The user who confirmed the above can easily recognize the position where the foreign object exists.

  In the display device according to the present invention, it is preferable that the image generation unit generates the defect area notification image in which the defect area image and the non-defective area image have different brightness or hue. In this way, by generating a defect area notification image in which the brightness or hue of the defect area image and the non-defect area image are different from each other and displaying them on the display screen, the user distinguishes between the defect area image and the non-defect area image. It becomes easy. As a result, the user can more easily recognize the position where the foreign object exists.

  Furthermore, in the display device according to the present invention, the display means combines a short wavelength color image or a plurality of short wavelength colors having different wavelengths when the imaging means receives light incident from the display screen. It is preferable to display an image of a different color on the display screen. Thus, when the imaging unit receives light incident from the display screen, the color of the wavelength that the imaging unit can receive with higher sensitivity is displayed on the display screen, so the imaging unit has higher sensitivity. Light of a color having a wavelength that can be received enters the imaging means from the display screen. Therefore, the contrast between the position where the foreign substance exists in the image data generated by the imaging unit and the other position becomes larger, and the detection accuracy of the foreign substance by the detection unit is improved.

  In the display device according to the present invention, it is preferable that the display means displays the image of the color with low brightness on the display screen. Thereby, it is possible to reduce power consumption when displaying an image of a predetermined color on the display screen while receiving light incident from the display screen by the imaging unit.

  Furthermore, in the display device according to the present invention, when the imaging unit receives light incident from the display screen, the display unit displays a color image including a wide range of wavelengths by combining a plurality of colors having different wavelengths. It is preferable to display on the display screen. In this way, by displaying an image of a color including a wider range of wavelengths on the display screen, an image close to white is displayed on the display screen. As a result, the imaging unit can receive light incident from the display screen with higher sensitivity, and the contrast between the position where the foreign substance is present and other positions in the image data generated by the imaging unit is further increased. Therefore, the detection means can detect even finer foreign matter or foreign matter with high light transmittance with high accuracy.

  In the display device according to the present invention, it is preferable that the detection unit detects a position of an object to be detected existing in a predetermined region in the image data. According to the above configuration, since a range in which the presence of a foreign object is expected can be determined in advance as a detection area, and the detection unit can detect the foreign object only in the area, power consumption can be reduced.

  Further, in the display device according to the present invention, the detection means calculates an average luminance of the pixels in the image data, and a difference between the luminance of the pixels and the average luminance is larger than a predetermined threshold value. In this case, it is preferable to determine that the pixel is a defective pixel and detect a region including the defective pixel as the position of the detection object. Thereby, the position where the foreign substance exists in the display screen can be accurately detected.

  In the display device according to the present invention, the threshold value is determined in advance according to the light transmittance of the detection object, and the detection means detects the detection target when the threshold value is lower. The object is determined to be a highly light-transmitting deposit, and the image generating means determines that the detected object is a highly light-transmitting deposit based on the determination result of the detecting means. It is preferable to generate a defect area notification image including an image to be shown.

  According to said structure, the detection means can detect the position of a foreign material more correctly according to the property of a foreign material, and can determine the property of the detected foreign material. Then, the image generation unit generates a defect area notification image including an image for notifying the user of the property of the foreign matter determined by the detection unit, and displays the image on the display screen so that the user can appropriately select the defect according to the property of the foreign matter. Foreign matter can be removed.

  For example, the detection means determines that the foreign matter detected when the threshold is lower is oil or fat such as a highly light-transmitting fingerprint, and the image generation means indicates a character indicating that the foreign matter is oil and fat together with the location of the oil or fat. A defect area notification image including a column is generated. Then, the display means displays the image on the display screen, so that the user can recognize that the foreign matter is oil and fat, and can wipe off the oil and fat by wiping the display screen because it is cloth.

  In the display device according to the present invention, the imaging unit receives light incident from the display screen after the display unit displays the defect area notification image on the display screen, and is based on the received light. Flaw detection image data is generated, and the detection means detects the position of the detection object present on the display screen based on the flaw detection image data, and the position of the detection object is the image. When the position of the detected object detected based on the data is the same as the detected object, it is determined that the detected object is a permanent attachment, and the image generating means is based on the determination result of the detecting means, It is preferable to generate a defect area notification image including an image indicating that the detected object is a permanent deposit.

  According to the above configuration, after displaying the defect area notification image on the display screen and notifying the user of the position of the foreign object, the imaging unit receives the light incident from the display screen again and generates the flaw detection image data. To do. The detecting means detects the position where the foreign substance exists based on the flaw detection image data, and the detected position of the foreign substance is the same as the position of the foreign substance detected based on the image data generated immediately before by the imaging means. In this case, it is determined that the foreign object cannot be removed by the user despite the notification of the position of the foreign object, that is, a permanently attached foreign object such as a scratch. And based on this determination result, an image generation means produces | generates the defect area notification image containing the image which shows that a permanent deposit | attachment exists on a display screen.

  Thus, by notifying the user of the position of the detected foreign object and then detecting the foreign object again at the same position, by performing a two-stage foreign object detection process that determines that the foreign object is a scratch or the like The foreign matter that can be removed and the foreign matter that is difficult to remove can be distinguished, and the user can be notified of the nature of the detected foreign matter.

  In the display device according to the present invention, the imaging unit includes a plurality of light receiving elements that receive light incident from the display screen, for each pixel of the display screen, It is preferable that they are arranged along a position corresponding to one of the diagonal lines. In this way, by receiving light incident from the display screen by a plurality of light receiving elements, the detection rate for foreign matters such as fine pollen is improved.

  Furthermore, in the display device according to the present invention, it is preferable that the light receiving elements are arranged such that the diagonal lines in which the light receiving elements are arranged form an angle of 90 degrees between the adjacent pixels of the display screen. Thereby, interference between the light receiving elements between the respective pixels can be prevented, and occurrence of problems such as moire and flicker on the display screen can be reduced.

  In the display device according to the present invention, it is preferable that the imaging unit receives light incident from the display screen by at least one of the plurality of light receiving elements provided for one pixel. According to the above configuration, the light receiving elements used for receiving the light incident from the display screen are limited, and only the light receiving elements to be used can be turned on and the light receiving elements not to be used can be turned off, thereby reducing power consumption. Can do.

  The present invention is a display device having a display screen having a function of detecting an input operation input to the display screen, receiving light incident from the display screen, and image data based on the received light. Based on the image data generated by the imaging means, the detection means for detecting the position of the detected object present on the display screen, and the detection means detected on the display screen A defect area notification including a defect area image representing the position of the object to be detected and a non-defect area image having a display format different from the defect area image representing a position other than the object to be detected on the display screen. Since the image generation means for generating an image and the display means for displaying the defect area notification image on the display screen are provided, it is possible to accurately notify the user of the position of the foreign matter on the display screen. An ability by removing the foreign matter in the user is notified position, can be removed foreign matter on reliably display screen. As a result, it is possible to accurately detect the input operation input on the display screen.

It is a block diagram which shows the outline of the display apparatus which concerns on one Embodiment of this invention. 1 is an external view of a display device according to an embodiment of the present invention. It is a block diagram which shows the structure of a display part. (A)-(e) is a figure which shows the transition of the display screen of the display apparatus which concerns on one Embodiment of this invention. It is a sequence diagram which shows the flow of the dust detection process by the display apparatus which concerns on one Embodiment of this invention. It is a sequence diagram which shows the flow of the confirmation process which confirms execution of the flaw detection process by the display apparatus which concerns on one Embodiment of this invention. It is the schematic of the 1st table memorize | stored in the memory | storage part. It is a figure which shows the display screen of the display apparatus which concerns on other embodiment of this invention. (A)-(g) is a figure which shows the transition of the display screen of the display apparatus which concerns on other embodiment of this invention. It is a sequence diagram which shows the flow of a display setting change process in the display apparatus which concerns on other embodiment of this invention. It is the schematic of the 2nd table memorize | stored in the memory | storage part. (A)-(c) is a figure which shows the display screen of the display apparatus which concerns on other embodiment of this invention. (A)-(i) is a figure which shows the transition of the display screen of the display apparatus which concerns on other embodiment of this invention. It is the schematic of the 3rd table memorize | stored in the memory | storage part. (A)-(f) is a figure which shows the display screen of the display apparatus which concerns on other embodiment of this invention. (A)-(g) is a figure which shows the transition of the display screen of the display apparatus which concerns on other embodiment of this invention. It is the schematic of the 4th table memorize | stored in the memory | storage part. (A)-(d) is a figure which shows the transition of the display screen of the display apparatus which concerns on other embodiment of this invention. (A)-(c) is a figure which shows the transition of the display screen of the display apparatus which concerns on other embodiment of this invention. It is sectional drawing for demonstrating the principal part structure of an imaging part. It is a figure for demonstrating the relationship between a pixel and the light receiving element of an imaging part. It is a figure explaining arrangement | positioning of a light receiving element. (A)-(c) is a figure for demonstrating a defect area | region determination process.

[First Embodiment]
An embodiment of a display device and a display method according to the present invention will be described below with reference to FIGS. FIG. 1 is a block diagram illustrating an outline of a display device 1 according to an embodiment, FIG. 2 is an overview of the display device 1, and FIG. 3 is a configuration of a display unit 3 provided in the display device 1. It is a block diagram which shows the outline.

(Display device 1)
As shown in FIG. 1, the display device 1 includes a main control unit 2, a display unit (display unit) 3, a key input unit 4, an imaging unit (imaging unit) 5, a defective area determination unit (detection unit) 6, and image processing. A unit (image generating means) 7 and a storage unit 8. The display device 1 includes a display screen 10 and input keys 11 as shown in FIG. As shown in FIG. 3, the display unit 3 includes a histogram calculation unit 31, a threshold value calculation unit 32, an image correction unit 33, and a backlight control unit 34, and displays an image based on an instruction from the main control unit 2. The image data is displayed on the unit 35.

  The main control unit 2 controls each component provided in the display device 1. In the display device 1, the functions of the constituent members are controlled by the main control unit 2 unless otherwise specified. Based on the input operation input by the user to the display screen 10, the main control unit 2 instructs the imaging unit 5 to capture image data, or instructs the defective region determination unit 6 to determine a defective region. To do. In addition, the main control unit 2 instructs the display unit 3 to display the image acquired from the image processing unit 7 and the data read from the storage unit 8.

  The display unit 3 displays an image sent from the main control unit 2 on the display screen 10 based on an instruction from the main control unit 2. Further, the backlight for irradiating the display screen 10 with light is controlled according to the display contents displayed on the display screen 10. As illustrated in FIG. 3, the histogram calculation unit 31 of the display unit 3 calculates a histogram of luminance of the image from the image data transmitted from the main control unit 2. Further, the threshold value calculation unit 32 of the display unit 3 calculates the maximum value of the histogram calculated by the histogram calculation unit 31 as the threshold value. That is, the histogram is searched from the higher luminance, and the luminance at the location where the histogram first appears is calculated as the threshold value.

  The backlight control unit 34 of the display unit 3 controls the brightness of the backlight so that light having an output luminance corresponding to the threshold calculated by the threshold calculation unit 32 is emitted. The image correction unit 33 corrects the output luminance of the image data transmitted from the main control unit 2 according to the threshold value calculated by the threshold value calculation unit 32. The image display unit 35 drives the backlight under the control of the backlight control unit 34 and displays the image corrected by the image correction unit 33 on the display screen 10. Even when the apparent output luminance of the display screen 10 is reduced by adjusting the brightness of the backlight by the backlight control unit 34, the correction of the output luminance of the image data by the image correction unit 33 Since the apparent output luminance of the display screen 10 is improved, the input / output relationship on the display screen 10 is canceled and an image is displayed with an appropriate luminance.

  The key input unit 4 receives a user input operation and transmits an input signal representing the input operation to the main control unit 2. That is, an input operation input to the display screen 10 having a touch panel function or an input operation input to the input key 11 provided at a place other than the display screen 10 on the surface of the display device 1 is received, An input signal representing the input operation is transmitted to the main control unit 2. The input key 11 provided on the surface of the display device 1 other than the display screen 10 is not particularly limited as long as it can detect a user input operation and transmit it to the main control unit 2.

  The imaging unit 5 captures an object to be detected based on an instruction from the main control unit 2 and captures image data. The capturing of the image data by the imaging unit 5 is performed through the display screen 10 and the light reflected from the detection object on the display screen 10 and returning from the display screen 10 among the backlight light irradiated on the display screen 10. This is performed by receiving light incident from the display screen side including external light incident thereon and generating image data based on the received light. The imaging unit 5 includes a light receiving element 193 (FIG. 20) for receiving the incident light, and may include a plurality of the light receiving elements 193. Details of the arrangement and configuration of the light receiving element 193 provided in the imaging unit 5 will be described later.

  Here, the detected object to be imaged by the imaging unit 5 is foreign matter such as dust, dust, oil, and scratches present on the display screen 10, and image data obtained by imaging the foreign matter is An image sent to the defect area determination unit 6 to be used for foreign matter determination, and further sent to the image processing unit 7 to distinguish between the position where the foreign matter is present and another position, and the presence of the foreign matter Is used to generate a defect area notification image for notifying the user of the above. The imaging unit 5 detects a user's input operation by imaging a finger or the like of the user who performs an input operation via the display screen 10 and analyzing the captured image data by a contact detection unit (not shown). . The user's input operation on the display screen 10 is detected not only when the finger or the like directly touches the display screen 10 but also when the finger or the like is brought close to the display screen 10.

  The defective area determination unit 6 determines a pixel at a position where an object to be detected such as a foreign object exists as a defective pixel using the image data captured by the imaging unit 5, and detects a defective area including the defective pixel. The defective area determination unit 6 transmits this detection result to the main control unit 2. Details of the defect area detection processing by the defect area determination unit 6 will be described later.

  Based on the detection result of the defect area determination unit 6, the image processing unit 7 can easily distinguish between a defect area and an area other than the defect area (non-defect area) using the image data captured by the imaging unit 5. Thus, the image data is corrected to generate a defect area notification image. In the correction of the image data, the display format is made different between the defective area and the non-defective area based on an instruction from the main control unit 2, or a character string for notifying the presence of the defective area is added. Details of generation of the defect area notification image by the image processing unit 7 will be described later.

  The storage unit 8 stores various data used in the display device 1 in a readable manner. The data stored in the storage unit 8 includes a defective pixel determination area, a determination time, a threshold, a detection result display method, and the like. The data stored in the storage unit 8 is read based on an instruction from the main control unit 2, and the data is updated or added based on an instruction from the main control unit.

  Next, an embodiment of dust detection processing by the display device 1 will be described below with reference to FIGS. 4A to 4E are diagrams showing transition of screen display displayed on the display screen 10 of the display device 1. In the dust detection process, the display screen 10 is displayed as shown in FIGS. ). In the present embodiment, the display screen 10 has a touch panel function, and the display means and the input means are integrated. The same applies to other embodiments described later.

  First, in the display screen 10 shown in FIG. 4A, a dust detection tool is selected from the menu screen. Based on this selection input, the dust detection tool is displayed on the display screen 10 as shown in FIG. The menu screen is displayed. Next, “1. Dust detection” is selected on the menu screen of the dust detection tool shown in FIG. 4B. Based on this selection input, the main control unit 2 stores the processing mode information for dust detection in the storage unit. 8 and displayed on the display screen 10 as shown in FIG.

  When the normal mode shown in FIG. 4C is selected, the main control unit 2 drives the imaging unit 5 and the defective area determination unit 6 to perform dust detection processing. Here, as shown in FIG. 4C, a brief description of the selected menu may be displayed on the lower side of the display screen 10, and the same applies to other embodiments. is there.

  During the dust detection process, a predetermined display color stored in the storage unit 8 is displayed on the display screen 10 as shown in FIG. Here, since the user has selected dust detection in the normal mode, the display color is cyan. When the dust detection process is completed and dust is detected, a defect area notification image representing the detection result is displayed based on the detection result and the display method stored in the storage unit 8 as shown in FIG. The generated image is displayed on the display screen 10.

  In FIG. 4E, the detected dust is displayed with a dotted line at the position where the dust exists on the display screen 10 and is displayed in a display format different from other positions. It is possible to easily recognize the location of dust. Then, when the user removes dust from the position indicated by the dotted line on the display screen 10, it is possible to prevent a reduction in detection accuracy due to the touch panel function of the display screen 10.

  The flow of dust detection processing by the display device 1 will be described more specifically with reference to FIG. FIG. 5 is a sequence diagram showing a flow of dust detection processing by the display device 1. First, as described above, when the user selects execution of dust detection processing, the main control unit 2 reads parameters necessary for dust detection from the storage unit 8 (step S400). In the storage unit 8, these pieces of information are stored in a first table 60 (FIG. 7) described later. The parameters read in step S400 include a display color, a determination area, a determination threshold value, and a determination time.

  Next, in step S <b> 401, the main control unit 2 instructs the display unit 3 to display the display color read from the storage unit 8 on the display screen 10. Then, in a state where the display color read from the storage unit 8 is displayed on the display screen 10, the main control unit 2 instructs the imaging unit 5 to capture image data (step S402). For the image data captured by the imaging unit 5, the defect region determination unit 6 determines the presence or absence of dust as a defect region and its position in the determination region, determination threshold value, and determination time read from the storage unit 8 (step). S403).

  In step S404, when the defect area determination unit 6 notifies the main control unit 2 of the defect area determination result (Yes), in step S405, the notification content is a notification indicating that a defect area exists. Determine whether. In step S404, when the defect region determination unit 6 does not notify the main control unit 2 of the defect region determination result (No), step S404 is repeated until the defect region determination unit 6 notifies the defect region determination result.

  In step S405, when the determination result of the defect area from the defect area determination unit 6 indicates that the defect area exists (Yes), in step S406, the main control unit 2 displays the detected defect area information. Based on this, the display setting of the defect area after detection is read from the storage unit 8. On the other hand, when the determination result of the defect area from the defect area determination unit 6 indicates that the defect area does not exist (No) in step S405, the main control unit 2 determines that the defect area is not detected in step S411. Is read from the storage unit 8. Then, the main control unit 2 instructs the display unit 3 to display the display content read from the storage unit 8 when no defective area is detected on the display screen 10 (step S412), and ends the dust detection process (END). ).

  Next, the image processing unit 7 emphasizes the defect area based on the display setting read in step S406 and the defect area information detected by the defect area determination unit 6, that is, a defect area indicating the position of the defect area. A defect area notification image including an image and a non-defect area image representing the position of the non-defect area is created (step S407). In step S408, the main control unit 2 determines whether or not there is a notification that the defect area notification image has been generated from the image processing unit 7, and if there is a notification (Yes), the defect generated by the image processing unit 7 An area notification image is acquired, and further, display contents when a defective area is detected are read from the storage unit 8 (step S409). On the other hand, in step S408, when there is no notification that the defect area notification image has been created from the image processing unit 7 (No), step S408 is performed until the image processing unit 7 notifies that the defect area notification image has been created. repeat.

  In step S <b> 410, the main control unit 2 instructs the display unit 3 to display the defective area notification image acquired from the image processing unit 7 on the display screen 10 with the display content read from the storage unit 8. After the defect area notification image is displayed on the display screen 10 in step S410, in the process A, a confirmation process is performed as to whether or not to continue the flaw detection process. The confirmation process will be described below with reference to FIG. FIG. 6 is a sequence diagram showing a flow of confirmation processing for confirming execution of the flaw detection processing by the display device 1.

  In the confirmation process, first, in step S500, the main control unit 2 reads the display content when performing the scratch process confirmation from the storage unit 8. Then, the main control unit 2 causes the display unit 3 to display the read display content on the display screen 10 (step S501). At this time, the display screen 10 displays a selection button for receiving an operation input for the user to select whether or not to perform scratch processing. In step S502, when the user's operation input indicates that a flaw is detected (Yes), a flaw detection process is performed. On the other hand, if it is determined in step S502 that the user's operation input indicates that no flaw is detected, the process A ends without performing the flaw detection process.

  Next, the first table 60 stored in the storage unit 8 will be described with reference to FIG. FIG. 7 is a schematic diagram of the first table 60 stored in the storage unit 8. As shown in FIG. 7, in the first table 60, parameters at the time of foreign object detection, display settings of the defect area after detection, and display contents of the display screen 10 after detection are stored in association with the processing mode. Yes. The first table 60 stores dust detection mode, oil detection mode, and flaw detection mode information as processing modes. The dust detection mode is further divided into a normal mode, a high sensitivity mode, and a power saving mode. Each piece of information is stored.

  In the dust detection mode, the display color at the time of detection is cyan in the normal mode, white in the high sensitivity mode, and blue in the power saving mode. These dust detection modes differ only in display color, but the others are common. Here, each dust detection mode will be described.

  The normal mode is a mode for detecting normal dust having a diameter of about 50 μm to 150 μm, such as hair. The display screen 10 is set so that the light receiving element 193 of the imaging unit 5 can image the display screen 10 with high sensitivity by displaying cyan of a short wavelength by combining blue and green. At this time, the display unit 3 is normal power consumption, and is a mode for general use.

  The high sensitivity mode is a mode for detecting fine dust having a diameter of about 30 μm to 40 μm, such as cedar pollen. By displaying white, which is a color including a wide range of short wavelengths, medium wavelengths, and long wavelengths, on the display screen 10, the light receiving element 193 of the imaging unit 5 can receive light from the display screen 10 with the highest sensitivity. Set as follows. At this time, since the brightness of the display screen 10 is maximized, the power consumption of the display unit 3 is also maximized. This mode is intended to detect dust more accurately with a sufficient remaining battery level.

  The power saving mode is a mode for detecting relatively large dust such as dust. By displaying blue light with a short wavelength and low brightness on the display screen 10, setting is made so that the light receiving element 193 of the imaging unit 5 can capture the display screen 10 with normal sensitivity. At this time, the display screen 10 displays blue with low brightness. When the brightness of the display screen is low, the brightness of the display screen is reduced and at the same time the brightness of the display screen is corrected, so that the power consumption of the display unit 3 can be suppressed without affecting the display contents. Therefore, this mode is effective when the remaining battery level is low.

  In the present embodiment, the short wavelength, medium wavelength, and long wavelength colors are all intended to be in the visible light region, for example, the short wavelength color is blue, the medium wavelength color is green, for example, and the long wavelength color is long. An example of the color is red. Here, the short wavelength is, for example, in the range of about 380 nm to about 495 nm, the medium wavelength is, for example, in the range of about 495 nm to about 590 nm, and the long wavelength is, for example, in the range of about 590 nm to about 750 nm. However, those skilled in the art will readily understand that they are not strictly limited to each range. That is, the visible light range itself is not strictly defined, and even a color having a wavelength outside the above range can be used as a short wavelength color, a medium wavelength color, or a long wavelength color.

  In contrast to the dust detection mode, the oil / fat detection mode is a mode for detecting an adhering substance having high light transmittance such as a fingerprint, and the scratch detection mode cannot be easily removed such as a scratch. This is a mode for detecting permanently attached matter. Details of the oil detection mode and the flaw detection mode will be described later. Each of these processing modes is selected by the user, and after the foreign object detection processing in the selected mode, the display settings 1 to 3 associated with each mode, the display contents at the time of detecting the defect area, and the non-detection of the defect area The detection result is displayed on the display screen 10 with the display content or the display content of the scratch processing confirmation.

[Second Embodiment]
Other embodiments of the display device 1 will be described below with reference to FIGS. FIG. 8 is a diagram illustrating an example of the sub-screen display displayed on the display screen 10 in the present embodiment. FIGS. 9A to 9G are displayed on the display screen 10 when the sub-screen display is set. It is a figure which shows the transition of the screen display performed. FIG. 10 is a sequence diagram illustrating a flow of display setting change processing, and FIG. 11 is a schematic diagram of the second table 100 stored in the storage unit 8. As shown in FIG. 8, the present embodiment is different from the first embodiment only in that when a defect area where dust is present is detected, an image showing the position of the detected defect area is displayed on the child screen. The configuration of the display device 1 and basic foreign object detection processing are the same as those in the first embodiment. Therefore, in the present embodiment, only differences from the first embodiment will be described, and other details will be omitted.

  As shown in FIG. 8, in the present embodiment, the image processing unit 7 is configured to create a defective area notification image such that the defective area image is displayed in a child screen obtained by reducing the display screen 10. The defect area is displayed in the sub-screen as in the present embodiment when the defect area notification image that displays the defect area image at the position where the dust exists on the display screen 10 is created. This is particularly effective in the case where the existing display such as a character string or an input button for accepting a user operation input overlaps with the defective area and it is difficult for the user to confirm the defective area. At this time, when the sub-screen is displayed on the full screen, the defect area is displayed at a position on the sub-screen corresponding to the position where the defective area exists, so that the user who has confirmed the sub-screen has a foreign object. The position can be easily recognized.

  Next, transition of the screen display displayed on the display screen 10 when the defect area is set to be displayed on the small screen will be described with reference to FIGS. First, in the display screen 10 shown in FIG. 9A, a dust detection tool is selected from the menu screen. Based on this selection input, the dust detection tool is displayed on the display screen 10 as shown in FIG. The menu screen is displayed. Next, on the menu screen of the dust detection tool shown in FIG. 9B, “3. Various settings” is selected, and on the basis of this selection input, as shown in FIG. The menu screen for various settings is displayed.

  When “1. display setting 1” is selected on the display screen shown in FIG. 9C, based on this selection input, as shown in FIGS. 9D and 9F, the display screen 10 The menu screen of display setting 1 is displayed. The display setting 1 menu is used to select either full screen or window display as shown in the first table 60 of FIG. As shown in (d) of FIG. 9, when “1. Full screen display” is selected on the menu screen of the display setting 1, the display screen 10 includes a defective area as shown in (e) of FIG. The image is displayed on the full screen, and an image including a character string and a selection button for confirming whether or not to change the setting to the full screen display is displayed. On the other hand, as shown in (f) of FIG. 9, when “2. Window display” is selected on the menu screen of the display setting 1, a defective area is displayed on the display screen 10 as shown in (g) of FIG. A sub-screen is displayed, and an image including a character string and a selection button for confirming whether or not to change the setting is displayed on the sub-screen display.

  The flow of the display setting changing process as shown in FIGS. 9A to 9G will be described with reference to FIGS. First, in step S <b> 900, the main control unit 2 reads the display contents and the reference diagram associated with the display setting selected by the user from among the display settings 1 from the second table 100 stored in the storage unit 8. That is, when the user selects “1. Full screen display”, the display contents and the reference diagram associated with “Full screen” of the display setting 1 are read from the second table 100 and the user selects “2. Window display”. Is selected, the display contents and the reference diagram associated with the “window” of the display setting 1 are read out.

  In step S901, the main control unit 2 causes the display unit 3 to display the read display content and the reference diagram on the display screen 10. At this time, the display screen 10 displays a selection button for accepting an operation input for the user to select whether or not to change the display setting (“Yes” and “No”). In step S902, when the operation input of the user indicates that the display setting is changed (Yes), the display setting 1 of the first table 60 in the storage unit 8 is changed to the display setting selected by the user (step S903). Then, the display setting change process is terminated (END). On the other hand, in step S902, when the user operation input indicates that the display setting is not changed (No), the display setting changing process is ended without changing the display setting 1 of the first table 60 of the storage unit 8. (END).

  In the present embodiment, the user changes the setting of the display setting 1, but in the display device 1, when the defective area and the screen layout are compared and the defective area overlaps with other existing displays, It may be determined that the defective area is displayed in a small screen and the display setting is automatically changed.

[Third Embodiment]
Another embodiment of the display device 1 will be described below with reference to FIGS. FIGS. 12A to 12C are diagrams illustrating an example of a screen display displayed on the display screen 10 in the present embodiment. FIGS. 13A to 13I are diagrams illustrating the display screen 10 when the display setting is changed. FIG. 14 is a schematic diagram of the third table 130 stored in the storage unit 8. As shown in FIGS. 12A to 12C, in the present embodiment, the brightness of the defect area when a defect area in which a foreign substance exists is detected is displayed so as to be different from the brightness other than the defect area. The configuration of the display device 1 and the basic foreign object detection processing are the same as those of the first embodiment. Therefore, in the present embodiment, only differences from the first embodiment will be described, and other details will be omitted.

  As shown in FIG. 12A, in this embodiment, the image processing unit 7 changes the lightness of the defective area to black and creates an image to be displayed different from the lightness (white) of the area other than the defective area. It is supposed to be. Further, as shown in FIG. 12B, the image processing unit 7 changes the brightness of the area other than the defective area to black, and creates an image to be displayed different from the brightness (white) of the defective area. It may be. Furthermore, as shown in FIG. 12C, the image processing unit 7 changes the brightness of the defect area and the surrounding area surrounding the defect area to black, and displays an image displayed differently from the brightness (white) of the other areas. You may come to create. Thus, by displaying the defect area and the area other than the defect area so that the brightness is different, the user can easily distinguish the defect area from the other area, and the visibility is improved.

  Next, the transition of the screen display displayed on the display screen 10 when the display setting is made such that the brightness is displayed differently in the defect area and the other area other than the defect area is shown in FIGS. This will be described with reference to i). First, on the display screen 10 shown in FIG. 13A, a dust detection tool is selected from the menu screen. Based on this selection input, the dust detection tool is displayed on the display screen 10 as shown in FIG. The menu screen is displayed. Next, on the menu screen of the dust detection tool shown in FIG. 13B, “3. Various settings” is selected, and on the basis of this selection input, as shown in FIG. The menu screen for various settings is displayed.

  When “2. Display setting 2” is selected on the display screen shown in FIG. 13C, based on this selection input, as shown in FIGS. 13D, 13F and 13H. The menu screen for display setting 2 is displayed on the display screen 10. The display setting 2 menu is for selecting one of patterns 1 to 3 as shown in the first table 60 of FIG. As shown in (d) of FIG. 13, when “1. pattern 1” is selected on the menu screen of display setting 2, as shown in (e) of FIG. 13, the brightness of the defect area is black, and An image in which the brightness of the area other than the defective area is white is displayed on the display screen 10 and includes a character string and a selection button for confirming whether or not to change the display setting so that the brightness of the defective area is displayed as black. An image is displayed.

  On the other hand, as shown in (f) of FIG. 13, when “2. Pattern 2” is selected on the menu screen of the display setting 2, the brightness of the defect area is white as shown in (g) of FIG. A character string and a selection for confirming whether or not the display setting is changed so that the brightness other than the defect area is displayed on the display screen 10 and the brightness other than the defect area is displayed as black. An image containing buttons is displayed.

  As shown in (h) of FIG. 13, when “3. Pattern 3” is selected on the menu screen of the display setting 2, as shown in (i) of FIG. Whether the display setting is changed so that the brightness of the area is black and the brightness of the other areas is white on the display screen 10 and the brightness of the defect area and the surrounding area is displayed as black. An image including a character string for confirming whether or not and a selection button is displayed.

  Note that the display setting changing process as shown in FIGS. 13A to 13I can be performed in the same manner as the flow shown in the sequence diagram shown in FIG. In step S901 in FIG. 10, the display content and the reference diagram associated with the display setting selected by the user among the display settings 2 are read from the third table 130 stored in the storage unit 8 illustrated in FIG. 14. This is different from the flow shown in FIG. Similarly, in step S903, the display setting 2 in the first table 60 is changed to the display setting selected by the user, which is different from the flow shown in FIG.

  Further, the image processing unit 7 may generate an image to be displayed so that the brightness of the defect area is different from the brightness of the area other than the defect area as described above, and the hue of the defect area may be an area other than the defect area. An image to be displayed different from the hue may be created. For example, the hue of the defective area is changed to blue, and an image to be displayed so as to be different from the hue (white) of the area other than the defective area is created. Further, for example, the hue other than the defect area is changed to yellow, and an image to be displayed different from the hue (black) of the defect area is created. As such display settings, pattern 1 and pattern 2 are stored in display setting 3 of first table 60, and display contents and reference diagrams associated with these patterns are stored in the same manner as third table 130 shown in FIG. The storage unit 8 may include the table. Thus, by displaying the defect area and the other area other than the defect area so that the hues are different, the user can easily distinguish the defect area from the other area, and the visibility is improved.

[Fourth Embodiment]
Another embodiment of the display device 1 will be described below with reference to FIGS. FIGS. 15A to 15F are diagrams illustrating examples of display images displayed on the display screen 10 during and after dust detection in the present embodiment, and FIGS. FIG. 17 is a diagram showing transition of screen display displayed on the display screen 10 when changing display settings during and after dust detection. FIG. 17 is a schematic diagram of the fourth table 160 stored in the storage unit 8. It is. This embodiment is different from the first embodiment only in that the determination area for dust detection processing is limited, and as shown in FIGS. 15A to 15F, only in the screen display of the display screen 10 during dust detection. The configuration of the display device 1 and basic foreign object detection processing are the same as those in the first embodiment. Therefore, in the present embodiment, only differences from the first embodiment will be described, and other details will be omitted.

  In the present embodiment, as shown in FIGS. 15A and 15B, the upper half of the display screen 10 is used as a dust detection processing determination area, and dust existing in the upper half of the display screen 10 is detected. At this time, the defective area determination unit 6 performs dust detection processing on only the upper half of the image of the display screen 10 captured by the imaging unit 5. Thereby, the power consumption of the display apparatus 1 can be reduced. At this time, the display unit 3 displays a predetermined display color only in the upper half that is the determination region of the display screen 10, or the image capturing unit 5 creates image data that images only the upper half that is the determination region of the display screen 10. This also reduces the power consumption of the display device 1.

  Further, as shown in FIGS. 15C and 15D, when the lower half of the display screen 10 is set as the determination area of the dust detection process and dust existing in the lower half of the display screen 10 is detected, and FIG. As shown in the middle (e) and (f), the left half or the right half of the display screen 10 is used as a dust detection processing determination region, and dust existing in the left half or the right half of the display screen 10 is also detected. A similar effect can be obtained.

  Next, transition of the screen display displayed on the display screen 10 when changing the setting of the determination area for dust detection processing will be described with reference to FIGS. First, on the display screen 10 shown in FIG. 16A, a dust detection tool is selected from the menu screen. Based on this selection input, the dust detection tool is displayed on the display screen 10 as shown in FIG. The menu screen is displayed. Next, on the menu screen of the dust detection tool shown in FIG. 16B, “3. Various settings” is selected, and based on this selection input, as shown in FIG. The menu screen for various settings is displayed.

  When “4. Area setting” is selected on the display screen shown in FIG. 16C, the display screen 10 is displayed as shown in FIGS. 16D and 16F based on this selection input. The area setting menu screen is displayed. The area setting menu is used to select any of the entire area, the upper half, the lower half, the right half, and the left half shown in the determination area of the first table 60 in FIG. As shown in (d) of FIG. 16, when “1. All areas” is selected in the area setting menu screen, all areas of the display screen 10 are displayed as dust detection as shown in (e) of FIG. An image including a character string and a selection button for confirming whether or not to change the setting so as to become a determination area is displayed on the display screen 10. At this time, the areas other than the character string and the selection button on the display screen 10 are all displayed in the same display format.

  On the other hand, when “2. Upper half” is selected on the area setting menu screen as shown in FIG. 16F, the upper half area of the display screen 10 is shown in FIG. Is displayed on the display screen 10 including a character string and a selection button for confirming whether or not the setting is changed so as to become a dust detection determination region. At this time, the character string and the selection button are displayed in the lower half of the display screen 10, and the upper half is displayed in a display format different from the area other than the character string and the selection button in the lower half.

  Note that the display setting change processing as shown in FIGS. 16A to 16G can be performed in the same manner as the flow shown in the sequence diagram shown in FIG. In step S901 of FIG. 10, the point that the display contents and the reference diagram associated with the determination area selected by the user among the determination areas are read from the fourth table 160 stored in the storage unit 8 shown in FIG. This is different from the flow shown in FIG. Similarly, in step S903, the determination area of the first table 60 is changed to the determination area selected by the user, which is different from the flow shown in FIG.

[Fifth Embodiment]
Another embodiment of the display device 1 will be described below with reference to FIGS. FIGS. 18A to 18D are diagrams showing transitions of screen displays displayed on the display screen 10 during the oil / fat detection processing. In the present embodiment, the display device 1 is different from the first embodiment only in that an oil / fat detection process is performed.

  First, on the display screen 10 shown in FIG. 18A, a dust detection tool is selected from the menu screen. Based on this selection input, the dust detection tool is displayed on the display screen 10 as shown in FIG. The menu screen is displayed. Next, in the menu screen of the dust detection tool shown in FIG. 18B, “2. Oil detection” is selected, and based on this selection input, the main control unit 2 performs the imaging unit 5 and the defect area determination unit. 6 is driven to detect oil and fat.

  During the oil detection process, white, which is the display color stored in the first table 60 of the storage unit 8, is displayed on the display screen 10, as shown in FIG. When the detection process of fats and oils is completed and fats and oils are detected, based on the detection results and the display method stored in the first table 60, as shown in FIG. An image is generated, and the generated image is displayed on the display screen 10. And in FIG.18 (d), when a user selects the confirmation button which shows having confirmed the detection of fats and oils, a fats and oils detection process is complete | finished.

  In FIG. 18 (d), the detected fats and oils are indicated by dotted lines at positions where the fats and oils are present on the display screen 10, and are displayed in a display color different from other positions. The presence of fats and oils can be easily recognized. Then, the user can prevent a reduction in detection accuracy due to the touch panel function of the display screen 10 by removing oil from the position indicated by the dotted line on the display screen 10.

  Such an oil / fat detection mode is a form for detecting an adhering substance having a high light transmittance such as a fingerprint which is usually difficult to see. By setting the display screen 10 to display white including a wide range of wavelengths, the light receiving element 193 of the imaging unit 5 is set so that the display screen 10 can be imaged with the highest sensitivity. The determination threshold at the time of detection is set to an average luminance of ± 25 or more in the dust detection mode, whereas in the oil detection mode, the average threshold is decreased to 5 to 24 or −24 to −5. It is set so that an adhering substance having a high light transmittance can be detected. Therefore, it is limited to detection of oil and fat having a small luminance change, and dust having a large luminance change is not detected.

[Sixth Embodiment]
Another embodiment of the display device 1 will be described below with reference to FIGS. FIGS. 19A to 19C are diagrams showing transition of screen display displayed on the display screen 10 during the flaw detection processing. In the present embodiment, the display device 1 is different from the other embodiments described above only in that the scratch detection process is performed after the dust detection process or the oil / fat detection process. The screen display shown in FIG. 19A shows that the user has confirmed the detection of dust or fat after the dust detection process in FIG. 4E or after the fat detection process in FIG. Is displayed on the display screen 10 in order to confirm whether or not to perform further scratch detection processing.

  When the user selects to detect a wound, based on the selection input, the main control unit 2 causes the imaging unit 5 to image the display screen 10 and generate scratch detection image data. During the flaw detection process, white, which is the display color stored in the first table 60 of the storage unit 8, is displayed on the display screen 10 as shown in FIG. Then, the defect area determination unit 6 detects the position where the foreign matter exists based on the flaw detection image data, and the imaging unit 5 determines the position of the detected foreign matter in the dust detection process or the oil / fat detection process performed immediately before. It is determined whether or not the position of the foreign object is the same as that detected based on the generated image data.

  When the determination results are the same, the defect area determination unit 6 notifies the user of the position of the foreign object, but the user cannot remove the foreign object, that is, a permanently attached foreign object such as a scratch. It is determined that When the scratch detection process is completed and a scratch is detected, the detection result indicating the detection of the scratch is shown based on the detection result and the display method stored in the first table 60, as shown in FIG. An image is generated and displayed on the display screen 10. Then, in FIG. 19C, when the user selects a confirmation button indicating that the detection of the flaw has been confirmed, the flaw detection process ends.

  In FIG. 19C, the detected scratch is indicated by a dotted line at the position where the scratch exists on the display screen 10 and is displayed in a display color different from the other positions. The presence of a flaw can be easily recognized. The user can prevent the detection accuracy from being reduced by the touch panel function of the display screen 10 by removing the scratch from the position indicated by the dotted line on the display screen 10.

  The flaw detection mode is a form in which a permanently attached deposit that cannot be easily removed, such as a flaw, is detected. By setting the display screen 10 to display white including a wide range of wavelengths, the light receiving element 193 of the imaging unit 5 is set so that the display screen 10 can be imaged with the highest sensitivity. The determination threshold value at the time of detection is set to ± 25 or more as in the dust detection mode, and is performed after the dust detection process or the oil detection process, so that the user cannot remove the deposit by the dust detection process or the oil detection process. Is determined as a “scratch” which is a permanent deposit.

[Seventh Embodiment]
Another embodiment of the display device 1 will be described below with reference to FIGS. FIG. 20 is a cross-sectional view for explaining the main configuration of the imaging unit 5 of the display device 1, and FIG. 21 is for explaining the relationship between the pixels of the display screen 10 and the light receiving elements 193 of the imaging unit 5. FIG. In the present embodiment, the main configuration of the imaging unit 5 will be described, and the other configurations are the same as those of the other embodiments described above, and thus detailed description thereof will be omitted.

  As shown in FIG. 20, the imaging unit 5 of the display device 1 includes a protection plate 190, a glass plate 191, a color filter 192, a light receiving element 193, and a glass plate 194 from the detected object 196 side to be imaged. In order. The surface of the protective plate 190 constitutes the display screen 10 of the display device 1, and the imaging unit 5 and the display unit 3 are integrally formed. The backlight 195 irradiates the display screen 10 on the surface of the protective plate 190 with light from the glass plate 194 side under the control of the backlight control unit 34 of the display unit 3.

  The light emitted from the backlight 195 to the display screen 10 is reflected by the detected object 196, and the reflected light is received by the light receiving element 193, thereby capturing the image data of the detected object on the display screen 10. When detecting an input operation to the display screen 10 by the user, the image data captured by the imaging unit 5 is subjected to image processing to determine, for example, contact or non-contact of the user's finger or the like with the display screen 10. And when it determines with a contact, the input operation by a user is detected by detecting the coordinate which contacted. When the display device 1 detects dust, oil or flaws or scratches, the defect area determination unit 6 extracts dust, oil or flaws or flaws as defective areas from the image data captured by the imaging unit 5 from the light receiving element 193.

  In the imaging unit 5 of the present embodiment, a plurality of light receiving elements 193 are provided for one pixel unit 197 of the display screen 10 as shown in FIG. For example, the display resolution of the display screen 10 is 480 × 640 pixels, the screen size is 2.4 inches (diagonal distance), and the size per pixel is 76.5 μm × 76.5 μm. At this time, if six light receiving elements 193 per pixel are arranged along the diagonal line of the pixel unit 197, the distance between the light receiving elements 193 is 18 μm. Therefore, as shown in the enlarged view of the pixel unit 197, at least two light receiving elements 193 are separated from the detected object 200 even if the detected object 200 has a diameter of about 30 μm to 40 μm and is as small as pollen, for example. Light can be received, so that minute dust can be detected.

  In addition, all six light receiving elements 193 arranged per pixel may be driven, or a part of the light receiving elements 193 may be driven according to a dust detection mode (normal mode, high sensitivity mode, or power saving mode) selected by the user. You may control to drive. For example, in the normal mode, every other light receiving element 193 shown in FIG. 21 is driven from one end. Thereby, since the distance between the light receiving elements 193 to be driven is 36 μm, dust having a size of 72 μm or more can be detected. Therefore, dust such as hair having a diameter of about 50 μm to about 150 μm can be detected. Since only half the light receiving elements 193 are driven as compared with the case where all six light receiving elements 193 are driven, the power consumption can be reduced to half.

  On the other hand, in the high sensitivity mode, by driving all six light receiving elements 193 per pixel, it is possible to detect smaller dust having a diameter of about 30 μm to about 40 μm as described above. Further, in the power saving mode, it is possible to detect dust having a larger diameter like dust while driving one light receiving element 193 per pixel to reduce power consumption.

  The light receiving element 193 used in the present embodiment is sensitive to blue or cyan, which is a combination of blue and green, and has the best sensitivity to white. Accordingly, when detecting dust, oil or flaws, the color displayed on the display screen 10 is set according to the required detection accuracy, and the color of light incident on the light receiving element from the display screen side is suitable for detection by the light receiving element 193. So that it becomes a different color.

[Eighth Embodiment]
Another embodiment of the display device 1 will be described below with reference to FIG. FIG. 22 is a diagram illustrating the arrangement of the light receiving elements 193 of the imaging unit 5. In the present embodiment, the arrangement of the light receiving elements 193 will be described, and other configurations are the same as those of the other embodiments described above, and thus detailed description thereof will be omitted. As shown in FIG. 22, a plurality of light receiving elements 193 are arranged obliquely along a diagonal line with respect to one pixel unit 197.

  Then, with respect to a specific target pixel unit 197, pixel units 197 adjacent in the vertical and horizontal directions are respectively rotated by 90 degrees with respect to the target pixel unit 197. That is, between the adjacent pixel units 197, the light receiving elements 193 are arranged so that the diagonal lines where the light receiving elements 193 are arranged form an angle of 90 degrees. As a result, the influence of moire, flicker, etc. occurring on the display screen 10 is reduced.

[Ninth Embodiment]
Another embodiment of the display device 1 will be described below with reference to FIGS. FIGS. 23A to 23C are diagrams for explaining the defect area determination processing by the defect area determination unit 6. FIG. 23A illustrates an example of the image 221 acquired by the imaging unit 5. 23B is an enlarged view of the X region in FIG. 23A, and FIG. 23C is a cross-sectional view taken along the line AA ′ shown in FIG. 23B. In the present embodiment, details of the defect area determination processing by the defect area determination unit 6 will be described, and the other configurations are the same as those of the other embodiments described above, and thus detailed description thereof will be omitted.

  In FIG. 23A, when a foreign substance 220 such as dust exists in the image 221 acquired by the imaging unit 5, the luminance of the pixel 222 at the position where the foreign substance exists is as shown in FIG. , Lower than the brightness of other positions. Here, the defect area determination unit 6 first calculates the average luminance of the determination area in the image 221 captured by the imaging unit 5. Then, this average luminance is compared with the luminance value of each pixel in the image 221, and when the difference is larger than the threshold value, it is determined as a defective area and recorded in a memory (not shown). In FIG. 23C, a pixel at a position where the luminance is lower than the threshold line set at a position where the luminance is lower than the average luminance line is determined as a defective area. That is, the lowest luminance portion shown in FIG. 23B is determined as a defective area.

  This process is performed for a predetermined determination time of, for example, one frame or more. In the second and subsequent frames, only when a defective area is detected at the same pixel position continuously within the determination time, the pixel is determined as a defective area, and the obtained exclusive OR is recorded in the memory. The determination result of the defective area recorded in the memory is transmitted to the main control unit 2. The defective area determination unit 6 may include an average luminance calculation unit and a first frame memory for timing adjustment (not shown). When image data is transmitted from the imaging unit 5, the average luminance is calculated based on the image data. The average luminance calculated by the calculation unit and the image data temporarily stored in the first frame memory are compared in a comparator (not shown). And the determination result by the comparison in a comparator may be memorize | stored in the 2nd frame memory (not shown) for defect area preservation | save. In this case, the determination result stored in the second frame memory is transmitted to the main control unit 2.

  In addition to the light incident on the light receiving element 193 from the display screen 10 side, the light is incident on the light receiving element 193 after being reflected from dust, oil or scratches on the display screen 10 by being irradiated from the backlight 195. External light incident from the outside of 10 is included. When dust such as dust is present on the display screen 10, external light and backlight light are substantially reflected by the dust. Therefore, when the outside light is strong, in the image acquired by the imaging unit 5, the pixel at the position where the dust is present is darker than the pixel at the other position, and when the outside light is weak, the dust is present. The pixel at the position is brighter than the pixels at other positions.

  On the other hand, when oil or fat is present on the display screen 10, external light and backlight light are substantially transmitted through the oil and fat. Therefore, when the external light is strong, in the image acquired by the imaging unit 5, the pixel at the position where the oil is present is slightly darker than the pixel at the other position. A pixel at an existing position is slightly brighter than pixels at other positions. Since the absolute value of the luminance change of the image due to the presence of oil and fat is smaller than the absolute value of the luminance change of the image due to the presence of dust, the white color that increases the spectral sensitivity of the light receiving element 193 described above is displayed on the display screen 10. Is preferred.

  The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention. That is, a form in which the display device 1 is used as a display device such as a mobile phone, a PC, a TV, a music player, a digital photo frame, and a book viewer is also included in the technical scope of the present invention.

  The present invention accurately notifies the user of the position of the foreign matter on the display screen, and by removing the foreign matter at the position notified by the user, the foreign matter on the display screen can be reliably removed. The input operation that has been input can be accurately detected, and can be suitably used for various electronic devices having a display screen having a touch panel function.

DESCRIPTION OF SYMBOLS 1 Display apparatus 2 Main control part 3 Display part (display means)
4 Key input section 5 Imaging section (imaging means)
6 Defect area determination unit (detection means)
7 Image processing unit (image generation means)
8 Storage unit 10 Display screen 193 Light receiving element 197 Pixel unit

Claims (14)

A display device having a display screen having a function of detecting an input operation input to the display screen,
Imaging means for receiving light incident from the display screen and generating image data based on the received light;
Detecting means for detecting the position of an object present on the display screen based on the image data generated by the imaging means;
A defect area image representing the position of the detected object detected by the detecting means on the display screen, and a position other than the detected object on the display screen, the defect area image being a display format Image generation means for generating a defect area notification image including different non-defect area images;
A display device comprising: display means for displaying the defect area notification image on the display screen.   The display device according to claim 1, wherein the image generation unit generates the defect area notification image so that the defect area image is displayed at a position of the detected object on the display screen. . The image generation means includes
The defect area notification image includes a sub-screen image obtained by reducing the display screen, and the defect area image is displayed at a position corresponding to the position of the detected object on the display screen in the sub-screen. The display device according to claim 1, wherein the display device is generated.   The display according to any one of claims 1 to 3, wherein the image generation means generates the defect area notification image in which the defect area image and the non-defect area image have different brightness or hue. apparatus.   The display means displays an image of a short wavelength color or a color image combining a plurality of short wavelength colors having different wavelengths on the display screen when the imaging means receives light incident from the display screen. The display device according to claim 1, wherein the display device is a display device.   The display device according to claim 5, wherein the display unit displays the image of the color with low brightness on the display screen.   The display means is characterized in that when the imaging means receives light incident from the display screen, an image of a color including a wide range of wavelengths combined with a plurality of colors having different wavelengths is displayed on the display screen. The display device according to any one of claims 1 to 4.   The display device according to claim 1, wherein the detection unit detects a position of an object to be detected existing in a predetermined region in the image data.   The detection means calculates an average luminance of a pixel in the image data, and if the difference between the luminance of the pixel and the average luminance is larger than a predetermined threshold value, the pixel is a defective pixel. The display device according to claim 1, wherein an area including the defective pixels is detected as a position of an object to be detected. The threshold value is predetermined according to the light transmittance of the object to be detected,
The detection means determines that the detected object detected when the threshold value is lower is an adhering substance having high light transmittance,
The image generation means generates a defect area notification image including an image indicating that the detected object is an adhering material having high light transmittance based on a determination result of the detection means. Item 10. The display device according to Item 9. The imaging means receives light incident from the display screen after the display means displays the defect area notification image on the display screen, generates scratch detection image data based on the received light,
The detection means detects the position of the detected object existing on the display screen based on the flaw detection image data, and the detected position detected based on the image data. When the position of the object is the same, it is determined that the detected object is a permanent deposit,
The said image generation means produces | generates the defect area notification image containing the image which shows that the said to-be-detected object is a permanent deposit based on the determination result of the said detection means. The display device according to any one of 10.   The imaging means includes a plurality of light receiving elements that receive light incident from the display screen for each pixel of the display screen, and the light receiving elements extend along a position corresponding to one of the diagonal lines of the pixels. It is arrange | positioned, The display apparatus of any one of Claims 1-11 characterized by the above-mentioned.   13. The display device according to claim 12, wherein the light receiving elements are arranged so that diagonal lines in which the light receiving elements are arranged form an angle of 90 degrees between adjacent pixels of the display screen.   14. The display device according to claim 12, wherein the imaging unit receives light incident from the display screen by at least one of the light receiving elements provided in plurality for one pixel.

Images