JPWO2018197984A1 - Display system and moving object - Google Patents

Abstract

To provide a display system with a new configuration and a moving object. The display system includes an imaging device, a display device, a feature amount output circuit, an image processing circuit, and a server. The imaging device has a function of acquiring imaging data. The feature amount output circuit has a function of acquiring feature amount data of the imaging data. The database has correction data and detection data. The database has a function of outputting correction data to the image processing circuit according to the feature amount data. The database selects correction data in accordance with machine learning or matching or similarity of feature amount data. The image processing circuit has a function of generating image data by correcting imaging data based on the correction data. The display device has a function of performing display according to the image data.

Description

  One embodiment of the present invention relates to a display system and a moving object.

  2. Description of the Related Art Vehicles equipped with an imaging device for imaging information around a vehicle and a display device for displaying information obtained by imaging are widely used (for example, Patent Document 1).

JP 2017-5678 A

  In a situation where the periphery of the vehicle is imaged during night driving, a light source (glare) of high illuminance by a headlight (headlight) of the vehicle approaches. Therefore, there is a problem that the obtained imaging data has excessive exposure around the light source, and the display of the area corresponding to the periphery of the light source becomes white. On the other hand, by suppressing the exposure, although the exposure around the light source can be adjusted, the displayed image becomes dark, and it is difficult to grasp the size and shape of the entire vehicle. That is, the obtained information is limited.

  An object of one embodiment of the present invention is to provide a novel display system, a mobile object, and the like.

  Alternatively, one embodiment of the present invention provides a novel display system, a moving object, and the like that can perform normal display even when a defect (unclear) portion occurs in acquired imaging data due to an imaging environment. This is one of the issues. Another object of one embodiment of the present invention is to provide a novel display system, a moving object, and the like which can improve visibility.

  In a display system according to one embodiment of the present invention, in imaging data obtained by an imaging device, a feature amount is extracted from part of information on a clearly imaged subject, and the corresponding correction data is selected by matching in a database. Therefore, the gist of the present invention is to correct an unclear area of the imaging data based on the correction data.

  One embodiment of the present invention includes an imaging device, a display device, a feature amount output circuit, an image processing circuit, and a database, the imaging device has a function of outputting imaging data, and the feature amount output circuit includes The database has correction data and detection data, and has a function of outputting correction data to the image processing circuit in accordance with the characteristic amount data; The image processing circuit has a function of generating image data by correcting imaging data based on the correction data, and the display device is a display system having a function of performing display according to the image data.

  In one embodiment of the present invention, it is preferable that the database has a function of selecting detection data that matches or is similar to the feature amount data and outputting correction data corresponding to the detection data to the image processing circuit.

  In one embodiment of the present invention, the database can update detection data serving as a weight parameter by machine learning using the feature data as learning data, and infer correction data that matches or is similar to the feature data. Display systems with functions are preferred.

  One embodiment of the present invention includes an imaging device, a display device, a feature amount output circuit, an image processing circuit, and a transmission / reception circuit; the imaging device has a function of acquiring imaging data; An image processing circuit having a function of acquiring feature amount data of imaging data, and a function of transmitting the feature amount data to a database having correction data and detection data via a transmission / reception circuit. Has a function of receiving image data for correction from a database via a transmission / reception circuit, and generating image data by correcting image data based on the image data for correction, and the display device has a function corresponding to the image data. It is a moving object having a function of performing display.

  In one embodiment of the present invention, the correction data is data corresponding to the detection data, and the detection data is preferably a moving object that matches or is similar to the feature amount data.

  In one embodiment of the present invention, the correction data is obtained by inference by inputting the feature data in a database in which detection data serving as weight parameters is updated by machine learning using the feature data as learning data. A moving object that is data is preferable.

  Note that other embodiments of the present invention are described in "Embodiments for Carrying Out the Invention" and "Drawings" described below.

  One embodiment of the present invention can provide a novel display system, a moving object including the display system, and the like.

  Alternatively, one embodiment of the present invention provides a novel display system, a moving object, and the like that can perform normal display even when a defect (unclear) portion occurs in acquired imaging data due to an imaging environment. be able to. Alternatively, one embodiment of the present invention can provide a novel display system, a moving object, and the like that can improve visibility.

FIG. 2 is a block diagram illustrating a display system. FIG. 2 is a block diagram illustrating a display system. FIG. 3 is a conceptual diagram illustrating a display system. FIG. 2 is a block diagram illustrating a display system. FIG. 2 is a block diagram illustrating a display system. 5 is a flowchart illustrating a display system. 5 is a flowchart illustrating a display system. FIG. 2 is a block diagram illustrating a display system. FIG. 3 is a diagram illustrating a display system. FIG. 3 is a diagram illustrating a display system. FIG. 9 is a diagram for explaining a specific example of the operation of the display system. FIG. 9 is a diagram for explaining a specific example of the operation of the display system. FIG. 9 is a diagram for explaining a specific example of the operation of the display system. FIG. 9 is a diagram for explaining a specific example of the operation of the display system. FIG. 9 is a diagram for explaining a specific example of the operation of the display system. FIG. 4 illustrates a configuration example of a semiconductor device. FIG. 4 illustrates a configuration example of a memory cell. FIG. 3 is a diagram illustrating a configuration example of an offset circuit. Timing chart. The figure which shows the example of application of a multiply-subtract operation circuit. FIG. 13 is a top view illustrating one embodiment of a display device. FIG. 4 is a cross-sectional view illustrating one embodiment of a display device. FIG. 4 is a cross-sectional view illustrating one embodiment of a display device. FIG. 9 illustrates a display device in a moving object. FIG. 4 illustrates an example of a moving object.

  Hereinafter, one embodiment of the present invention will be described with reference to the drawings. However, one embodiment of the present invention can be implemented in many different modes, and it is easily understood by those skilled in the art that the mode and details can be variously changed without departing from the spirit and scope thereof. Is done. Therefore, the present invention is not construed as being limited to the following description.

(Embodiment 1)
<Display system configuration>
A display system of one embodiment of the present invention will be described.

  FIG. 1 is a block diagram illustrating an example of a display system which is one embodiment of the present invention. The display system 10 illustrated in FIG. 1 includes an imaging device 11, a feature amount output circuit 12, a database 13, an image processing circuit 14, and a display device 15.

  The imaging device 11 is, specifically, a camera module attached to a moving body such as an automobile. The imaging device 11 has a function of outputting imaging data 16. The imaging data 16 is output to the feature amount output circuit 12 and the image processing circuit 14.

  It is preferable that the imaging device 11 includes an imaging element having a wide dynamic range. For example, by using the imaging device 11 including an imaging element having selenium, when capturing an image of a subject having a large difference in brightness, it is possible to reduce unclear portions of image data.

  The feature amount output circuit 12 extracts a feature amount in the imaging data 16 and outputs it as feature amount data. It is effective to extract the feature amount by combining image data obtained by the imaging device 11 with data obtained from a sensor or the like used for detecting a relative speed or a relative position of a subject.

  The feature amount includes an appearance feature amount (appearance feature amount). For example, the external appearance feature amount of a car includes, for example, a vehicle body color of the car, a license plate number, a car type, a car width, a car height, a turn signal lamp, a headlight, and the like. The feature amount data of these feature amounts is output to the database 13 as encoded data or data cut out from the original imaging data.

  By default, the feature amount is set so as to be easily detected by image recognition, so that the load of image recognition processing can be reduced. Since the turn-on or blink of the blinker lamp or the brake lamp changes depending on the situation, it is also effective to exclude the blinker lamp and the brake lamp from the targets of image recognition.

  The unclear area of the imaging data 16 is an area in which it is difficult to extract a feature amount because the contrast ratio between light and dark is small. Therefore, the feature amount output circuit 12 can reduce the data in the unclear region of the imaging data by extracting the feature amount.

  The feature amount output circuit 12 extracts, for example, a Haar-like feature, a Histograms of Oriented Gradients (HOG) feature, a Scald Invariance Feature Transform (SIFT) feature, or a local extraction technique for a SURF (Speeded UpRobust) feature. It is also effective to extract and output the feature amount data of a desired portion by combining. It is also effective to smooth the image data 16 with a Gaussian filter and to select information on characteristic regions.

  It is also effective to perform object detection by combining a local gradient feature extraction technique and SVM (Support Vector Machine) learning to extract and output feature amount data of a desired portion. The feature amount output circuit 12 may output, for example, feature amounts automatically calculated by CNN (Convolutional Neural Network) learning as feature amount data.

  The feature amount output circuit 12 is mainly configured as a microcomputer, and includes a processor, a memory, an I / O, and a bus connecting these. In the feature amount output circuit 12, various functions can be realized by a processor executing a program stored in a memory in advance based on information of various sensors in addition to the imaging device 11.

  The database 13 stores feature amount data 17, correction data 18, and detection data 19. The correction data 18 is data for correcting the imaging data 16. The detection data 19 is data for selecting the correction data 18 based on the feature amount data.

  The correction data 18 is data for correcting the imaging data 16 in the image processing circuit 14. It is preferable that the correction data 18 be data that can be displayed with a higher resolution than the imaging data 16. With this configuration, the resolution of an image obtained by correcting the imaging data 16 can be increased, and the visibility of display on the display device 15 can be increased.

  Note that the image data obtained by correcting the imaging data using the correction data 18 may be different from an image obtained by actual visual observation. For example, image data may be partially different in a vehicle type, a body shape, and the like. Alternatively, correction data may be selected from subjects having the same feature amount data. Even in such a case, since the entire information of the subject can be learned as compared with the image data without correction, the situation can be grasped more efficiently by the projected image.

  The detection data 19 is data from which correction data 18 can be selected based on the feature data 17 input from the feature output circuit 12. For example, the detection data can be feature amount data such as an appearance feature amount extracted based on the correction data 18. With this configuration, desired correction data 18 can be selected by selecting detection data 19 that matches or is similar to the feature amount data 17.

  Alternatively, the detection data 19 may be data serving as a weight parameter in CNN learning. In this case, the CNN learning in the database 13 is performed by using, as learning data, data obtained by adding a correct answer label corresponding to the correction data 18 to the feature amount data 17 input from the feature amount output circuit 12 collected in advance as learning data. This is performed by updating the data 19. The detection data 19 is updated by repeating the CNN learning. The accuracy can be improved by repeating the updating. By using the CNN using the updated detection data 19 as a weight parameter, the correction data 18 corresponding to the input of the feature amount data 17 can be selected with high accuracy.

  The database 13 has a function of storing the correction data 18 and the detection data 19, a memory for storing a program for selecting the correction data 18 corresponding to the characteristic amount data 17, and a processor for executing the program. And has a function as a computer having In the database 13, various functions can be realized by executing a program stored in the computer in advance.

  The image processing circuit 14 is a circuit having a function of correcting data in an unclear area of the imaging data 16 based on the correction data 18 and outputting the corrected imaging data as image data. Note that the unclear area of the imaging data 16 corresponds to an area where it is difficult to extract a feature amount, such as an area having a small contrast ratio between light and dark. For this reason, a configuration in which an area in which extraction of a feature amount is difficult is corrected based on the correction data 18 can be given as an example. The correction data 18 is particularly effective when correcting the entire contour of the subject in the imaging data 16.

  The image processing circuit 14 is preferably configured to hold the once acquired correction data. With this configuration, once the correction data 18 is held, it can be repeatedly used for correcting the imaging data.

  It is preferable that the correction data 18 has a higher resolution than the imaging data 16. When the resolution of the imaging data obtained by the imaging device 11 is smaller than the resolution of the display, the correction data 18 can be used for the super-resolution processing of the imaging data, and the corrected and displayed image becomes clearer. be able to.

  The display device 15 performs display based on the image data generated by the image processing circuit. The display device 15 can be used as a substitute for a rearview mirror in the case of an automobile. As the display device 15, in addition to a liquid crystal display device, a self-luminous display device having an electroluminescence element or the like can be used.

  The display system 10 has an imaging device 11 and a display device 15. The display system 10 extracts a feature amount from a part of the information of the subject in the imaging data 16 obtained by the imaging device 11, selects the correction data 18 by collation in the database 13, and selects the correction data 18. The unclear area of the image data 16 is corrected based on the original data.

  The display system 10 is preferably applied to a mobile body such as an automobile. Specifically, it is particularly suitable in a situation where an image of the periphery of a car is captured by an imaging device such as a camera, and the state is visually recognized on a display device. Note that the moving object is a vehicle that moves like an automobile. Therefore, the moving object is not limited to a car, but includes a bus, a train, an airplane, and the like.

  In an automobile equipped with an imaging device and a display device as described above, an unclear area may occur in imaging data obtained by the imaging device depending on a change in a situation such as day or night, fine or rain. For example, headlights of a car behind or water droplets attached to an image pickup device cause an unclear area. When an unclear area exists in the obtained imaging data, it is difficult to grasp the size and shape of the entire subject, and the obtained information is limited. As described above, in the display system 10 according to one embodiment of the present invention, a feature amount is extracted from part of the information on the subject, and the data is selected in the database 13 to select the correction data 18. Correct the area. Therefore, it is effective in improving visibility.

  The application of the display system 10 of the present invention is not limited to a mobile object. This is effective when displaying image data captured by the imaging device on a display device. For example, it is effective when correcting an unclear portion of image data obtained by an image capturing device such as a security camera or a smartphone.

  When the database 13 is provided in a remote place remote from the automobile, the database 13 may be configured as shown in the block diagram of the display system shown in FIG. The display system 10A shown in FIG. 2 has a configuration in which a transmission / reception circuit 20 and a network 21 are added to the configuration of the display system 10 in FIG. That is, data output from the feature output circuit 12 to the database 13 and data output from the database to the image processing circuit 14 are transmitted via the transmission / reception circuit 20 and the network 21.

  The configuration of the display system 10A in FIG. 2 can be described with a conceptual diagram illustrated in FIG. That is, the display system 10A is configured to transmit / receive data to / from the database 13 via the relay station 24 from the automobile 22A to which the transmission / reception circuit 20 is attached.

  Note that the configuration of the display system 10 in FIG. 1 can be described with a conceptual diagram illustrated in FIG. That is, the imaging device 11, the feature amount output circuit 12, the database 13, the image processing circuit 14, and the display device 15 that constitute the display system 10 are configured to be provided in the automobile 22. The feature amount output circuit 12, the database 13, and the image processing circuit 14 are illustrated as a data processing circuit 23.

  Further, in the configuration of the display system 10 in FIG. For example, FIG. 4 illustrates a block diagram of a display system 10B including a plurality of imaging devices 11_1-11_n (n is a natural number) and display devices 15_1-15_n. In the configuration of FIG. 4, there are a plurality of pieces of data output by the feature amount output circuit 12, the database 13, and the image processing circuit 14, in addition to the plurality of pieces of image data 16 output by the imaging devices 11 </ b> _ <b> 1 to 11 — n. Note that the display device 15 can have a configuration in which image data obtained by imaging with the plurality of imaging devices 11_1-11_n is displayed side by side by having a large display portion.

  Further, in the configuration of the display system 10 in FIG. 1, the configuration may be such that the presence or absence of correction is switched according to the illuminance of the surroundings. For example, FIG. 5 illustrates a block diagram of a display system 10C in which a sensor 25 is added to the configuration of the display system 10 of FIG.

  In the configuration of the display system 10C in FIG. 5, the sensor 25 can control whether to stop the function of the feature amount output circuit 12 according to the output. For example, when the sensor 25 is an illuminance sensor, the output of the feature amount data from the feature amount output circuit 12 is stopped when the illuminance is large, and the output of the feature amount data from the feature amount output circuit 12 is started when the illuminance is small. can do.

  The switching of the output of the feature amount data from the feature amount output circuit 12 may be performed by turning on and off the switching function by the user. In this case, the configuration may be such that the function can be switched between start and stop according to an input from a touch sensor or the like.

  The sensor 25 may be configured to be used in combination with another sensor such as a millimeter wave radar that can be used to detect the relative speed and the relative position of the subject.

  In the display system of one embodiment of the present invention described above, even if a defect (unclear) portion occurs in the acquired imaging data due to the imaging environment, the imaging data is corrected based on the correction data, and a normal image is corrected. Image data for display can be generated. Therefore, the visibility of the image displayed on the display device can be improved.

<Operation of display system>
FIG. 6 is a flowchart for explaining an operation example of the display system 10 described in FIG.

  In the operation of the display system described with reference to FIG. 6, the feature data of the imaging data 16 is extracted to input the feature data to the database 13, and the detection data 19 that matches or is similar to the feature data 17 is compared. The correction data 18 is selected. Then, the selected correction data 18 is used by the image processing circuit 14 to correct the imaging data 16.

  First, step S11 is a step of acquiring imaging data 16 by the imaging device 11. The imaging data 16 is output to the feature amount output circuit 12. The same imaging data 16 is also output to the image processing circuit 14.

  Step S12 is a step of determining whether or not the input imaging data 16 has a display abnormality. Step S12 may be performed together with the feature amount extraction in step S13. When there is a display abnormality, it is partially difficult to extract a feature amount such as an appearance feature amount. Therefore, it is effective to determine a display abnormality based on a change in the feature amount data. Alternatively, the display abnormality may be determined according to the output of a sensor such as an illuminance sensor. The user may determine the display abnormality by looking at the display based on the imaging data 16 and perform switching by a sensor or the like. If it is determined that there is no display abnormality (No), the process proceeds to the step of displaying by the display device 15 in step S17. If it is determined that there is a display abnormality (Yes), the process proceeds to step S13.

  Step S13 is a step in which the feature amount output circuit 12 performs feature amount extraction. The obtained feature amount is output to the database 13 as feature amount data 17. The feature amount data 17 is data obtained by combining an appearance feature amount based on an image recognition process and a feature amount obtained by a local gradient feature extraction technique of a characteristic region.

  Step S14 is a step of selecting (searching) detection data 19 stored in the database 13 corresponding to the feature amount data obtained in step S13. The detection data 19 is data for selecting the correction data 18 based on the feature amount data 17 as described above.

  Step S15 is a step of causing the image processing circuit 14 to output correction data 18 corresponding to the detection data 19 that matches or is similar to (hit) the feature data 17 by the search in step S14. The correction data 18 is data for correcting a defective (unclear) area of the imaging data 16 as described above.

  Step S16 is a step of generating image data by correcting the imaging data 16. Image data obtained based on the corrected imaging data 16 is data in which excessive exposure around the light source and information on the size and shape of the entire subject have been corrected. That is, the obtained information can be complemented by the correction data 18.

  Step S17 is a step of performing display based on the image data on the display device 15.

  In the series of operations of the display system described with reference to FIG. 6, it is possible to select the correction data 18 by comparing the detection data 19 that matches or is similar to the feature amount data 17. The selected correction data 18 is used by the image processing circuit 14 to correct the imaging data 16, thereby generating image data for performing a normal display when a missing (unclear) portion occurs in the obtained imaging data. can do. Therefore, the visibility of the image displayed on the display device can be improved.

  In the operation of the display system, the feature amount of the imaging data 16 can be extracted by the database 13 and the correction data 18 can be selected from the feature amount data. The database 13 is trained so that the detection data 19 is used as a weight parameter in the CNN learning to infer the correction data 18 corresponding to the feature data. Then, the selected correction data 18 can be used by the image processing circuit 14 to correct the imaging data 16.

  FIG. 7 shows a flowchart for explaining an example of learning of the display system 10 when the detection data 19 is used as a weight parameter in CNN learning.

  In step S21, the same feature extraction as in step S13 is performed. That is, in step S12, the feature amount of the imaging data 16 having the display abnormality is extracted. The feature amount data obtained in step S21 is training data and test data, and the original imaging data is preferably different data, and it is preferable to generate and prepare a large amount of feature amount data.

  Step S22 is a step of giving a correct answer label to the feature amount data. The correct answer label corresponds to the correction data 18 optimal for correcting the imaging data 16 corresponding to the feature amount data.

  Step S23 is a step of providing the feature amount data for training and updating the detection data 19 serving as a weight parameter. It is effective to update the weight parameter by using a known learning method such as the error back propagation method.

  Step S24 is a step of inputting feature amount data. The feature amount data input in step S24 is test data with a correct label.

  A step S25 decides (Yes or No) whether or not the output signal obtained in the step S24 corresponds to the correct answer label given earlier. If not, the updating of the parameters in step S23 is repeated.

  Step S26 is a judgment (Yes or No) as to whether the accuracy of whether or not the correction data 18 matching or similar to the feature amount data 17 is obtained is equal to or less than the set value. If the accuracy is equal to or less than the set value, the updating of the parameters in step S23 is repeated.

  The detection data 19 is updated by repeating the CNN learning. By repeating the update, the accuracy of the correction data 18 corresponding to the input of the feature amount data 17 can be improved. By using the CNN using the updated detection data 19 as a weight parameter, the correction data 18 corresponding to the input of the unknown feature amount data 17 can be selected with high accuracy.

  FIG. 8 is a block diagram for explaining a configuration example of the CNN (convolutional neural network) described in FIG.

  The convolutional neural network 30 shown in FIG. 8 includes an input layer 31, an intermediate layer 32, and an output layer 33.

  FIG. 8 illustrates the feature amount data 17 as input data in the input layer 31.

  FIG. 8 illustrates a filter 34, convolution data 35, pooling data 36, filter 37, convolution data 38, and pooling data 39 in the intermediate layer 32. Note that the configuration of the intermediate layer 32 is an example, and a configuration in which the pooling process and the convolution operation process by the filters 34 and 37 are further performed in multiple layers, or a configuration in which operation processes such as padding and stride are performed may be employed.

  FIG. 8 shows the total combined data 40 and the output data 41 in the output layer 33. The output data 41 is obtained by processing the obtained all combined data 40 (y1-ym in the figure) with a softmax function, and obtaining an output corresponding to the correct answer label.

  In the CNN learning in the flowchart described with reference to FIG. 7, the parameters (weight data) of the filters 34 and 37 are updated, and the output data 41 (O (y1) -O (ym) in the figure) with the label is obtained. To be able to learn and reason. By inputting the feature amount data 17 to the convolutional neural network 30, it is possible to output the corresponding correction data 18 to the image processing circuit 14.

  A specific example of the display system 10 described above and its operation will be described. 9 and 10, an example in which the display system 10 is applied to an automobile will be described.

  FIG. 9A is a schematic view of the automobile 22 as viewed from behind. FIG. 9A illustrates a camera as the imaging device 11 and a room mirror as the display device 15.

  FIG. 9B illustrates the automobile 22 and another automobile 42 located behind the automobile 22. The automobile 42 illuminates a headlight as a light source 43 toward the automobile 22 in an illumination direction 44. FIG. 9B illustrates a state in which the imaging device 11 of the automobile 22 is imaging toward the automobile 42 in the imaging direction 45.

  Note that, in the example of FIG. 9B, the car 42 is illustrated as a subject to be imaged by the imaging device 11, but the subject is not limited to the car. For example, the subject may be any object such as a pedestrian, a bicycle, a street lamp, a sign, or the like, for which correction data can be selected from the image data by extracting a feature amount. In the case of a pedestrian, it is also effective to extract a feature amount from the feature of a part such as a foot or a hand and correct the part of the head or the upper body with the correction data. By displaying the image data of the whole pedestrian on the display device, the visibility can be improved.

  In the case of FIG. 9B, the imaging data 16 obtained by the imaging device 11 can be represented as shown in FIG. 9C. That is, as shown in FIG. 9C, the illuminance of the light source 43 increases. By suppressing the exposure, the shape of the area around the light source 43 is recognized, but it is difficult to recognize the shape of the entire vehicle 42. Therefore, as shown in FIG. 9D, when capturing an image of an automobile having a high illuminance light source during night driving or the like, a clear area 48 and an unclear area 49 are mixed in the captured data.

  As another example, FIG. 10A is a schematic diagram viewed from behind a car 22A. FIG. 10A illustrates a camera as the imaging device 11 </ b> A and a room mirror as the display device 15. FIG. 10A is an enlarged schematic view of the imaging device 11A, and illustrates the lens 46 and the water droplet 47. In the case of rain or the like, water droplets 47 may adhere to the surface of the lens 46, and the obtained image data may be unclear. Although FIG. 10A illustrates a configuration in which the display device 15 is arranged at the position of the rearview mirror, it is also effective for a configuration in which a display device at the position of the side mirror or the dashboard is arranged.

  FIG. 10B illustrates the automobile 22A and another automobile 42 located behind the automobile 22A. FIG. 10B illustrates a state in which the imaging device 11A of the automobile 22A is imaged toward the automobile 42 in the imaging direction 45.

  In the case of FIG. 10B, the imaging data 16 obtained by the imaging device 11 can be represented as shown in FIG. That is, as shown in FIG. 10C, the image of the automobile 42 becomes unclear due to the imaging of the water droplet 47, and it becomes difficult to recognize the shape of the entire automobile 42. Therefore, as shown in FIG. 10D, when an image of an automobile is taken during traveling on rainy weather or the like, a clear area 48 and an unclear area 49 are mixed in the image data.

  In the above-described display system 10 of one embodiment of the present invention, a feature amount is extracted from information of a part of the automobile 42 in the clear area 48 of the imaging data 16, and the feature amount data 17 is compared in the database 13 for correction. By selecting the data 18, it is possible to correct an unclear area of the imaging data 16.

  Therefore, in a vehicle equipped with the display system 10, even if an unclear area occurs in the image data obtained by the imaging apparatus in accordance with a change in a situation such as day or night, fine or rain, a corrected image can be visually recognized. For example, even if a headlight of a rear automobile or a water droplet attached to an imaging device causes an unclear area, the imaging data can be corrected by selecting correction data corresponding to the feature amount data. Therefore, it is easy to grasp the size and shape of the entire subject. Further, the obtained information can be increased, which is effective for improving the visibility.

<Specific example for explaining operation of display system>
FIG. 11 is a flowchart for explaining a more specific example of the operation of the display system. FIG. 11 illustrates a flow of data in the imaging device 11, the feature amount output circuit 12, the database 13, the image processing circuit 14, and the display device 15 in the display system 10 described in FIG.

  In step S31 shown in FIG. 11, acquisition of imaging data in the imaging device 11 and output of imaging data to the feature amount output circuit 12 are performed. In step S31 shown in FIG. 11, the database 13 stores the correction data and the detection data. The storage of the correction data 18 and the detection data 19 in the database 13 may be performed in advance.

  Step S31 shown in FIG. 11 can be illustrated as a block diagram shown in FIG. FIG. 12 illustrates the imaging data 16 illustrated in FIG. 9C for easy understanding. In FIG. 12, images 18A of a plurality of cars are shown as the correction data 18 for easy understanding. In FIG. 12, an image 19A of a headlight, which is a part of an image of an automobile, is illustrated as detection data for easy understanding.

  In step S32 shown in FIG. 11, the calculation of the feature data in the feature output circuit 12 and the output of the feature data from the feature output circuit 12 to the database 13 are performed.

  Step S32 shown in FIG. 11 can be illustrated as a block diagram shown in FIG. FIG. 13 shows an image 17A of a headlight as a part of an image of a car, which is an appearance feature amount, as the feature amount data 17, for easy understanding.

  Step S33 shown in FIG. 11 is a process for inputting the feature data 17 from the feature output circuit 12 to the database 13 and detecting the search data 19 that matches or is similar to the feature data using the headlight image 17A. The data 19 is selected, and the correction data 18 corresponding to the detection data 19 is output to the image processing circuit 14.

  Step S33 shown in FIG. 11 can be illustrated as a block diagram shown in FIG. In FIG. 14, in order to facilitate understanding, it is assumed that the feature amount data of the headlight image 17A matches the feature amount data of the headlight image 19A, and the correction data 18 output to the image processing circuit 14 is used. An image 18A is shown.

  In step S34 shown in FIG. 11, image data 16 from the imaging device 11 as correction source data and correction data 18 from the database 13 as correction data are input to the image processing circuit 14, The data 16 is corrected. The corrected imaging data is output from the image processing circuit 14 to the display device 15 as image data. The display device 15 receives image data from the image processing circuit 14 and can perform clear display.

  Step S34 shown in FIG. 11 can be illustrated as a block diagram shown in FIG. FIG. 15 illustrates the imaging data 16 illustrated in FIG. 9C for easy understanding. FIG. 15 illustrates an image 18 </ b> A as the correction data 18 output to the image processing circuit 14 for easy understanding. FIG. 15 illustrates image data 51 corresponding to image data output to the display device 15 for easy understanding.

  In the above-described display system 10 according to one embodiment of the present invention, as shown in FIG. 11, the feature data is extracted from a part of the information of the imaging data 16, and the feature 18 is selected, and an unclear area of the image data 16 is corrected. Therefore, it is effective in improving visibility.

  Therefore, in a vehicle equipped with the display system 10, even if an unclear area occurs in the image data obtained by the imaging apparatus in accordance with a change in a situation such as day or night, fine or rain, a corrected image can be visually recognized. For example, even if a headlight of a rear automobile or a water droplet attached to an imaging device causes an unclear area, the imaging data can be corrected by selecting correction data corresponding to the feature amount data. Therefore, it is easy to grasp the size and shape of the entire subject. Further, the obtained information can be increased, which is effective for improving the visibility.

(Embodiment 2)
In this embodiment, a configuration example of a semiconductor device that can be used for the neural network 30 described in the above embodiment will be described. In particular, it can be used for a product-sum operation circuit such as the intermediate layer 32 and output layer 33.

<Configuration example of semiconductor device>
FIG. 16 shows a configuration example of a semiconductor device MAC having a function of performing a neural network operation. The semiconductor device MAC can be used for at least a part of the neural network 30 described in the above embodiment. The semiconductor device MAC has a function of performing a product-sum operation of first data corresponding to the connection strength (weight) between neurons and second data corresponding to input data. Note that the first data and the second data can be analog data or multivalued data (discrete data), respectively. Further, the semiconductor device MAC has a function of converting data obtained by the product-sum operation using an activation function.

  The semiconductor device MAC includes a cell array CA, a current source circuit CS, a current mirror circuit CM, a circuit WDD, a circuit WLD, a circuit CLD, an offset circuit OFST, and an activation function circuit ACTV.

  The cell array CA has a plurality of memory cells MC and a plurality of memory cells MCref. FIG. 16 shows a memory cell MC (MC [1, 1] to [m, n]) in which the cell array CA has m rows and n columns (m, n is an integer of 1 or more) and m memory cells MCref (MCref). [1] to [m]). The memory cell MC has a function of storing first data. Further, the memory cell MCref has a function of storing reference data used for the product-sum operation. The reference data can be analog data or multi-valued data.

The memory cell MC [i, j] (i is an integer of 1 to m, j is an integer of 1 to n) is a wiring WL [i], a wiring RW [i], a wiring WD [j], and a wiring BL [J]. The memory cell MCref [i] is connected to the wiring WL [i], the wiring RW [i], the wiring WDref, and the wiring BLref. Here, the current flowing between the memory cell MC [i, j] and the wiring BL [ _j] is represented as IMC [i _{, j],} and the current flowing between the memory cell MCref [i] and the wiring BLref is _{IMCref [ i]} .

  FIG. 17 shows a specific configuration example of the memory cell MC and the memory cell MCref. FIG. 17 shows the memory cells MC [1,1] and [2,1] and the memory cells MCref [1] and [2] as representative examples, but the same applies to other memory cells MC and the memory cells MCref. Can be used. Each of the memory cell MC and the memory cell MCref has a transistor Tr11, Tr12, and a capacitor C11. Here, the case where the transistors Tr11 and Tr12 are n-channel transistors is described.

  In the memory cell MC, the gate of the transistor Tr11 is connected to the wiring WL, one of a source and a drain is connected to the gate of the transistor Tr12, and the first electrode of the capacitor C11, and the other of the source and the drain is connected to the wiring WD. Have been. One of a source and a drain of the transistor Tr12 is connected to a wiring BL, and the other of the source and the drain is connected to a wiring VR. The second electrode of the capacitor C11 is connected to the wiring RW. The wiring VR is a wiring having a function of supplying a predetermined potential. Here, as an example, a case where a low power supply potential (such as a ground potential) is supplied from the wiring VR is described.

  A node connected to one of the source and the drain of the transistor Tr11, the gate of the transistor Tr12, and the first electrode of the capacitor C11 is referred to as a node NM. Further, the nodes NM of the memory cells MC [1,1] and [2,1] are described as nodes NM [1,1] and [2,1], respectively.

  The memory cell MCref has the same configuration as the memory cell MC. Note that the memory cell MCref is connected to the wiring WDref instead of the wiring WD, and is connected to the wiring BLref instead of the wiring BL. In the memory cells MCref [1] and [2], the node connected to one of the source and the drain of the transistor Tr11, the gate of the transistor Tr12, and the first electrode of the capacitor C11 is referred to as a node NMref [1]. , [2].

The nodes NM and NMref function as holding nodes for the memory cell MC and the memory cell MCref, respectively. The node NM holds first data, and the node NMref holds reference data. In addition, currents I _{MC [1} , _1] and I _{MC [2, 1]} flow from the wiring BL [1] to the transistors Tr12 of the memory cells MC [1, 1] and [2, 1], respectively. Further, currents I _{MCref [1]} and I _{MCref [2]} flow from the wiring BLref to the transistors Tr12 of the memory cells MCref [1] and [2], respectively.

  Since the transistor Tr11 has a function of holding the potential of the node NM or the node NMref, the off-state current of the transistor Tr11 is preferably small. Therefore, it is preferable to use an OS transistor with extremely low off-state current as the transistor Tr11. Accordingly, a change in the potential of the node NM or the potential of the node NMref can be suppressed, and calculation accuracy can be improved. Further, the frequency of the operation of refreshing the potential of the node NM or the node NMref can be reduced, so that power consumption can be reduced.

  The transistor Tr12 is not particularly limited, and for example, a Si transistor, an OS transistor, or the like can be used. When an OS transistor is used as the transistor Tr12, the transistor Tr12 can be manufactured using the same manufacturing apparatus as the transistor Tr11, and manufacturing cost can be reduced. Note that the transistor Tr12 may be an n-channel type or a p-channel type.

The current source circuit CS is connected to the wirings BL [1] to [n] and the wiring BLref. The current source circuit CS has a function of supplying a current to the wirings BL [1] to [n] and the wiring BLref. Note that a current value supplied to the wirings BL [1] to [n] may be different from a current value supplied to the wiring BLref. Here, the current supplied from the current source circuit CS to the wirings BL [1] to [n] is denoted by I _C , and the current supplied from the current source circuit CS to the wiring BLref is denoted by I _Cref .

  The current mirror circuit CM has wirings IL [1] to [n] and a wiring ILref. The wirings IL [1] to [n] are connected to the wirings BL [1] to [n], respectively, and the wiring ILref is connected to the wiring BLref. Here, connection portions between the wirings IL [1] to [n] and the wirings BL [1] to [n] are referred to as nodes NP [1] to [n]. A connection point between the wiring ILref and the wiring BLref is referred to as a node NPref.

The current mirror circuit CM has a function to flow a current _{I CM} corresponding to the potential of the node NPref wiring ILref, the ability to flow to the current _{I CM} wiring IL [1] to [n]. Figure 16 is discharged current _{I CM} from the wiring BLref to the wiring ILref, wiring BL [1] to the wiring from the [n] IL [1] to [n] to the current _{I CM} is an example to be discharged . Further, the current flowing in the cell array CA via the wiring BL [1] to [n] from the current mirror circuit _CM, denoted as _I B [1] to [n]. Further, a current flowing from the current mirror circuit CM to the cell array CA via the wiring _BLref is referred to as _IBref .

  The circuit WDD is connected to the wirings WD [1] to [n] and the wiring WDref. The circuit WDD has a function of supplying a potential corresponding to the first data stored in the memory cell MC to the wirings WD [1] to WD [n]. The circuit WDD has a function of supplying a potential corresponding to reference data stored in the memory cell MCref to the wiring WDref. The circuit WLD is connected to the wirings WL [1] to WL [m]. The circuit WLD has a function of supplying a signal for selecting a memory cell MC or a memory cell MCref to which data is to be written to the wirings WL [1] to WL [m]. The circuit CLD is connected to the wirings RW [1] to RW [m]. The circuit CLD has a function of supplying a potential corresponding to the second data to the wirings RW [1] to RW [m].

The offset circuit OFST is connected to the wirings BL [1] to [n] and the wirings OL [1] to [n]. The offset circuit OFST detects a current amount flowing from the wirings BL [1] to [n] to the offset circuit OFST and / or a change amount of a current flowing from the wirings BL [1] to [n] to the offset circuit OFST. Having. Further, the offset circuit OFST has a function of outputting a detection result to the wirings OL [1] to OL [n]. Note that the offset circuit OFST may output a current corresponding to the detection result to the wiring OL, or may convert a current corresponding to the detection result into a voltage and output the voltage to the wiring OL. The current flowing between the cell array CA and the offset circuit OFST is denoted by I _α [1] to [n].

  FIG. 18 shows a configuration example of the offset circuit OFST. The offset circuit OFST illustrated in FIG. 18 includes circuits OC [1] to OC [n]. Each of the circuits OC [1] to OC [n] includes a transistor Tr21, a transistor Tr22, a transistor Tr23, a capacitor C21, and a resistor R1. The connection relation of each element is as shown in FIG. Note that a node connected to the first electrode of the capacitor C21 and the first terminal of the resistor R1 is referred to as a node Na. A node connected to the second electrode of the capacitor C21, one of the source and the drain of the transistor Tr21, and the gate of the transistor Tr22 is referred to as a node Nb.

  The wiring VrefL has a function of supplying the potential Vref, the wiring VaL has a function of supplying the potential Va, and the wiring VbL has a function of supplying the potential Vb. The wiring VDDL has a function of supplying the potential VDD, and the wiring VSSL has a function of supplying the potential VSS. Here, the case where the potential VDD is a high power supply potential and the potential VSS is a low power supply potential is described. The wiring RST has a function of supplying a potential for controlling the conduction state of the transistor Tr21. A source follower circuit is formed by the transistor Tr22, the transistor Tr23, the wiring VDDL, the wiring VSSL, and the wiring VbL.

  Next, an operation example of the circuits OC [1] to OC [n] will be described. Here, an operation example of the circuit OC [1] will be described as a typical example; however, the circuits OC [2] to OC [n] can be operated similarly. First, when the first current flows through the wiring BL [1], the potential of the node Na becomes a potential corresponding to the first current and the resistance value of the resistor R1. At this time, the transistor Tr21 is on, and the potential Va is supplied to the node Nb. Thereafter, the transistor Tr21 is turned off.

Next, when a second current flows through the wiring BL [1], the potential of the node Na changes to a potential corresponding to the second current and the resistance value of the resistor R1. At this time, since the transistor Tr21 is off and the node Nb is in a floating state, the potential of the node Nb changes due to capacitance coupling with a change in the potential of the node Na. Here, assuming that the change in the potential of the node Na is ΔV _Na and the capacitance coupling coefficient is 1, the potential of the node Nb is Va + ΔV _Na . When the threshold voltage of the transistor Tr22 and _{V th,} the potential Va + ΔV _Na -V _th is output from the wiring OL [1]. Here, by setting Va = _Vth , the potential ΔV _Na can be output from the wiring OL [1].

The potential ΔV _Na is determined according to the amount of change from the first current to the second current, the resistance element R1, and the potential Vref. Here, since the resistance element R1 and the potential Vref are known, the amount of change in the current flowing through the wiring BL from the potential ΔV _Na can be obtained.

  The signal corresponding to the amount of current detected by the offset circuit OFST and / or the amount of change in current as described above is input to the activation function circuit ACTV via the wirings OL [1] to [n].

  The activation function circuit ACTV is connected to the wirings OL [1] to [n] and the wirings NIL [1] to [n]. The activation function circuit ACTV has a function of performing an operation for converting a signal input from the offset circuit OFST according to a predefined activation function. As the activation function, for example, a sigmoid function, a tanh function, a softmax function, a ReLU function, a threshold function, and the like can be used. The signal converted by the activation function circuit ACTV is output to the wirings NIL [1] to NIL [n] as output data.

<Operation example of semiconductor device>
Using the above-described semiconductor device MAC, a product-sum operation of the first data and the second data can be performed. Hereinafter, an operation example of the semiconductor device MAC when performing the product-sum operation will be described.

FIG. 19 shows a timing chart of an operation example of the semiconductor device MAC. FIG. 19 illustrates the wiring WL [1], the wiring WL [2], the wiring WD [1], the wiring WDref, the node NM [1,1], the node NM [2,1], and the node NMref [1] in FIG. , node Nmref [2], wiring RW [1], and changes in potentials of the wiring RW [2], the current _{I B [1] -I α [} 1], and shows a transition of the value of the current _{I Bref} . Current _{I B [1] -I α [} 1] , the wiring BL [1] from the memory cell MC [1, 1], which corresponds to the sum of the current flowing through the [2,1].

  Here, the operation will be described focusing on the memory cells MC [1,1] and [2,1] and the memory cells MCref [1] and [2] shown in FIG. 17 as representative examples. The MC and the memory cell MCref can be operated similarly.

[Storage of First Data]
First, the time at T01-T02, the potential of the wiring WL [1] becomes high level, the _V PR _{-V W [1,1]} greater potential the potential of the wiring WD [1] is higher than the ground potential (GND), wiring potential of WDref becomes the _{V PR} greater potential than the ground potential. Further, the potentials of the wiring RW [1] and the wiring RW [2] become the reference potential (REFP). Note that the potential _{VW [1,1]} is a potential corresponding to the first data stored in the memory cell MC [1,1]. The potential _VPR is a potential corresponding to the reference data. Accordingly, the transistor Tr11 included in the memory cell MC [1,1] and the memory cell MCref [1] is turned on, and the potential of the node NM [1,1] becomes V _PR −V _{W [1,1]} and the node NMref The potential of [1] becomes _VPR .

At this time, the current IMC _{[1,1], 0} flowing from the wiring BL [1] to the transistor Tr12 of the memory cell MC [1,1] can be expressed by the following equation. Here, k is a constant determined by the channel length, channel width, mobility, capacity of the gate insulating film, and the like of the transistor Tr12. _Vth is a threshold voltage of the transistor Tr12.

The current I _{MCref [1], 0} flowing from the wiring BLref to the transistor Tr12 of the memory cell MCref [1] can be expressed by the following equation.

  Next, at a time T02 to T03, the potential of the wiring WL [1] is at a low level. Accordingly, the transistor Tr11 included in the memory cell MC [1,1] and the memory cell MCref [1] is turned off, and the potentials of the nodes NM [1,1] and NMref [1] are held.

  Note that as described above, an OS transistor is preferably used as the transistor Tr11. Accordingly, leakage current of the transistor Tr11 can be suppressed, and the potentials of the nodes NM [1,1] and NMref [1] can be accurately held.

Then, at time T03-T04, the potential of the wiring WL [2] becomes the high level, the potential of the wiring WD [1] becomes _V PR _{-V W [2,1]} greater potential than the ground potential, of the wiring WDref potential becomes the V _PR greater potential than the ground potential. Note that the potential _{VW [2,1]} is a potential corresponding to the first data stored in the memory cell MC [2,1]. Accordingly, the transistor Tr11 included in the memory cell MC [2,1] and the memory cell MCref [2] is turned on, and the potential of the node NM [2,1] becomes V _PR −V _{W [2,1]} and the node NMref The potential of [2] becomes _VPR .

At this time, the current IMC _{[2,1], 0} flowing from the wiring BL [1] to the transistor Tr12 of the memory cell MC [2,1] can be expressed by the following equation.

The current I _{MCref [2], 0} flowing from the wiring BLref to the transistor Tr12 of the memory cell MCref [2] can be expressed by the following equation.

  Next, at a time T04 to T05, the potential of the wiring WL [2] is at a low level. Accordingly, the transistor Tr11 included in the memory cell MC [2,1] and the memory cell MCref [2] is turned off, and the potentials of the nodes NM [2,1] and NMref [2] are held.

  By the above operation, the first data is stored in the memory cells MC [1,1] and [2,1], and the reference data is stored in the memory cells MCref [1] and [2].

Here, currents flowing through the wiring BL [1] and the wiring BLref from time T04 to time T05 are considered. A current is supplied from the current source circuit CS to the wiring BLref. The current flowing through the wiring BLref is discharged to the current mirror circuit CM and the memory cells MCref [1] and [2]. If the current supplied from the current source circuit CS to the wiring BLref is I _Cref , and the current discharged from the wiring BLref to the current mirror circuit CM is I _{CM, 0} , the following equation is established.

The current from the current source circuit CS is supplied to the wiring BL [1]. The current flowing through the wiring BL [1] is discharged to the current mirror circuit CM and the memory cells MC [1,1], [2,1]. Further, current flows from the wiring BL [1] to the offset circuit OFST. If the current supplied from the current source circuit CS to the wiring BL [1] is I _{C, 0} , and the current flowing from the wiring BL [1] to the offset circuit OFST is Iα _{, 0} , the following equation is established.

[Multiply-sum operation of first data and second data]
Next, at a time T05 to T06, the potential of the wiring RW [1] becomes V _{X [1]} higher than the reference potential. At this time, the potential VX _[1] is supplied to the capacitor C11 of each of the memory cell MC [1,1] and the memory cell MCref [1], and the potential of the gate of the transistor Tr12 increases due to capacitive coupling. Note that the potential Vx _[1] is a potential corresponding to the second data supplied to the memory cell MC [1,1] and the memory cell MCref [1].

The amount of change in the potential of the gate of the transistor Tr12 is a value obtained by multiplying the amount of change in the potential of the wiring RW by a capacitance coupling coefficient determined by the configuration of the memory cell. The capacitance coupling coefficient is calculated based on the capacitance of the capacitor C11, the gate capacitance of the transistor Tr12, the parasitic capacitance, and the like. Hereinafter, for the sake of convenience, the description will be given on the assumption that the amount of change in the potential of the wiring RW and the amount of change in the potential of the gate of the transistor Tr12 are the same, that is, the capacitance coupling coefficient is 1. In fact, it may be determined potential V _x in consideration of the capacitive coupling coefficients.

When the memory cell MC [1] and the potential _{V X} in the capacitor C11 of the memory cell MCref [1] _[1] is supplied, the node NN [1] and the node Nmref [1] _{V X} voltage of each _[1] To rise.

Here, from time T05 to T06, the current IMC _{[1,1], 1} flowing from the wiring BL [1] to the transistor Tr12 of the memory cell MC [1,1] can be expressed by the following equation.

In other words, by supplying the potential _{V X [1]} to the wiring RW [1], the current flowing from the wiring BL [1] to the transistor Tr12 of the memory cell MC [1, 1] _is, ΔI _{MC [1,1]} = _{IMC [1,1], 1−} IMC _{[1,1], 0 is} increased.

In addition, from time T05 to T06, a current I _{MCref [1], 1} flowing from the wiring BLref to the transistor Tr12 of the memory cell MCref [1] can be expressed by the following equation.

In other words, by supplying the potential _{V X [1]} to the wiring RW [1], the current flowing from the wiring BLref to the transistor Tr12 of the memory cell MCref [1] _{is, ΔI MCref [1] = I} MCref [1], 1 -I _{MCref [1], 0 is} increased.

Further, current flowing through the wiring BL [1] and the wiring BLref is considered. The current I _Cref is supplied to the wiring BLref from the current source circuit CS. The current flowing through the wiring BLref is discharged to the current mirror circuit CM and the memory cells MCref [1] and [2]. Assuming that the current discharged from the wiring BLref to the current mirror circuit CM is ICM _{, 1} , the following equation is established.

The wiring BL [1], the current _{I C} is supplied from the current source circuit CS. The current flowing through the wiring BL [1] is discharged to the current mirror circuit CM and the memory cells MC [1,1], [2,1]. Further, current flows from the wiring BL [1] to the offset circuit OFST. Assuming that the current flowing from the wiring BL [1] to the offset circuit OFST is Iα _{, 1} , the following equation is established.

Then, from equation (E1) to the formula (E10), the current I _{alpha, 0} and the current I _{alpha, 1} of the difference (differential current [Delta] I _alpha) can be represented by the following equation.

Thus, the differential current [Delta] I _alpha, a value corresponding to the product of _{V X [1]} and the potential _{V W [1,1].}

  After that, at times T06 to T07, the potential of the wiring RW [1] becomes the ground potential, and the potentials of the nodes NM [1,1] and NMref [1] are the same as those of times T04 to T05.

Then, at time _{T07-T08, V X [1} ] than the potential reference potential of the wiring RW [1] becomes large potential, and _{V X [2]} greater than the potential is the reference potential of the wiring RW [2] Become. Accordingly, the potential VX _[1] is supplied to each of the capacitor elements C11 of the memory cell MC [1,1] and the memory cell MCref [1], and the nodes NM [1,1] and NMref [ 1] increases by _{VX [1]} . Further, the potential VX _[2] is supplied to each of the capacitor elements C11 of the memory cell MC [2,1] and the memory cell MCref [2], and the nodes NM [2,1] and NMref [2 _] are coupled by capacitive coupling. ] Increases by V _{X [2]} .

Here, from time T07 to time T08, the current IMC _{[2,1], 1} flowing from the wiring BL [1] to the transistor Tr12 of the memory cell MC [2,1] can be expressed by the following equation.

In other words, by supplying the potential _{V X [2]} to the wiring RW [2], the current flowing from the wiring BL [1] to the transistor Tr12 of the memory cell MC [2,1] is, ΔI _{MC [2,1]} = _{IMC [2,1], 1−} IMC _{[2,1], 0 is} increased.

Further, from time T05 to time T06, a current I _{MCref [2], 1} flowing from the wiring BLref to the transistor Tr12 of the memory cell MCref [2] can be expressed by the following equation.

That is, by the wiring RW [2] supplying a potential _{V X [2],} the current flowing from the wiring BLref to the transistor Tr12 of the memory cell MCref [2] _{is, ΔI MCref [2] = I} MCref [2], 1 -I _{MCref [2], 0 is} increased.

Further, current flowing through the wiring BL [1] and the wiring BLref is considered. The current I _Cref is supplied to the wiring BLref from the current source circuit CS. The current flowing through the wiring BLref is discharged to the current mirror circuit CM and the memory cells MCref [1] and [2]. Assuming that the current discharged from the wiring BLref to the current mirror circuit CM is ICM _{, 2} , the following equation is established.

The wiring BL [1], the current _{I C} is supplied from the current source circuit CS. The current flowing through the wiring BL [1] is discharged to the current mirror circuit CM and the memory cells MC [1,1], [2,1]. Further, current flows from the wiring BL [1] to the offset circuit OFST. Assuming that the current flowing from the wiring BL [1] to the offset circuit OFST is Iα _{, 2} , the following equation is established.

Then, formula (E1) to the formula (E8), and, from equation (E12) to the formula (E15), the current I _{alpha, 0} and the current I _{alpha, 2} of the difference (differential current [Delta] I _alpha) is expressed by the following equation be able to.

Thus, the differential current [Delta] I _alpha, add to the potential _{V W [1,1]} and the product of the potential _{V X [1],} the product of the potential _{V W [2,1]} and the potential _{V X [2],} the The value depends on the combined result.

  After that, at times T08 to T09, the potentials of the wirings RW [1] and [2] become the ground potential, and the potentials of the nodes NM [1,1] and [2,1] and the nodes NMref [1] and [2] become It becomes the same as time T04-T05.

As shown in equation (E9) and formula (E16), the differential current [Delta] I _alpha inputted to the offset circuit OFST, the potential _{V X} which corresponds to the first data (weights), a second data (input data ) corresponding to a value corresponding to the combined result plus the product of the potential V _W. That is, by measuring the differential current [Delta] I _alpha offset circuit OFST, it is possible to obtain the result of the product sum calculation of the first and second data.

  Although the above description focuses on the memory cells MC [1,1] and [2,1] and the memory cells MCref [1] and [2], the numbers of the memory cells MC and the memory cells MCref may be arbitrarily set. Can be. The difference current ΔIα when the number m of rows of the memory cells MC and the rows of the memory cells MCref is an arbitrary number can be expressed by the following equation.

  Also, by increasing the number n of columns of the memory cells MC and the memory cells MCref, the number of product-sum operations executed in parallel can be increased.

  As described above, by using the semiconductor device MAC, the product-sum operation of the first data and the second data can be performed. Note that by using the configuration illustrated in FIG. 17 as the memory cell MC and the memory cell MCref, a product-sum operation circuit can be configured with a small number of transistors. Therefore, the circuit scale of the semiconductor device MAC can be reduced.

  When the semiconductor device MAC is used for the operation in the neural network, the number m of rows of the memory cells MC corresponds to the number of input data supplied to one neuron, and the number n of columns of the memory cells MC corresponds to the number of neurons. Can be. For example, consider a case where a product-sum operation is performed using the semiconductor device MAC in the intermediate layer. At this time, the number m of rows of the memory cells MC is set to the number of input data supplied from the input layer (the number of neurons of the input layer), and the number n of columns of the memory cells MC is set to the number of neurons of the intermediate layer. Can be set.

  As described above, by using the semiconductor device MAC, the product-sum operation of the neural network can be performed. Further, by using the memory cell MC and the memory cell MCref shown in FIG. 17 for the cell array CA, it is possible to provide an integrated circuit capable of improving operation accuracy, reducing power consumption, or reducing the circuit scale. .

Incidentally, by the potential V _X which corresponds to the first data (weight) is to the case where more than an arbitrary reference potential "positive", the smaller "negative", it is possible to perform a subtract operations. Therefore, the semiconductor device MAC can function as a product difference operation circuit.

  Using the multiply-subtract operation circuit, it is possible to remove suddenly generated noise. As an example, FIGS. 20A and 20B are shown. FIGS. 20A and 20B are images showing a vehicle traveling in a dark place such as in a tunnel. FIG. 20A shows that although the vehicle 110 located outside the tunnel can be visually recognized due to the effect of the headlight 105 of the vehicle 100 that is suddenly turned on, the visibility of the pedestrian 120 and the pedestrian 130 in the tunnel is reduced. FIG.

  FIG. 20B illustrates an image processing using a product difference calculation circuit, in which the influence of the suddenly turned on headlight 105 is reduced, so that not only the vehicle 110 located outside the tunnel but also the pedestrian 120 in the tunnel is shown. And a state in which the visibility of the pedestrian 130 is also increased. Note that the difference information between the current frame image and the immediately preceding frame image may be acquired by the product difference calculation circuit, and image processing may be performed using the difference information.

  2. Description of the Related Art In recent years, in-vehicle image sensors have been expected for automatic driving and as side mirrors and rearview mirrors. By combining one embodiment of the present invention and a mechanism for adjusting sensitivity for each pixel, an obstacle or a person can be accurately detected even in a situation where the light of another vehicle is dazzling.

  This embodiment can be combined with any of the other embodiments as appropriate.

(Embodiment 3)
In this embodiment, an example of the display device 15 described in the above embodiment will be described below with reference to FIGS.

  FIG. 21 is a top view illustrating a display device 700 applicable to the display device 15 described in the above embodiment. A display device 700 illustrated in FIG. 21 includes a pixel portion 702 provided over a first substrate 701, a demultiplexer 703, a source driver 704, a gate driver 706 provided over the first substrate 701, a pixel portion 702, The semiconductor device includes a sealant 712 provided so as to surround the demultiplexer 703 and the gate driver 706, and a second substrate 705 provided so as to face the first substrate 701. Note that the first substrate 701 and the second substrate 705 are sealed with a sealant 712. That is, the pixel portion 702, the demultiplexer 703, and the gate driver 706 are sealed with the first substrate 701, the sealant 712, and the second substrate 705. Although not shown in FIG. 21, a display element is provided between the first substrate 701 and the second substrate 705.

  In the display device 700, the pixel portion 702, the demultiplexer 703, the source driver 704, and the gate driver 706 are provided in regions different from the region surrounded by the sealant 712 on the first substrate 701. Is provided with an FPC terminal unit 708 (Flexible printed circuit). An FPC 716 is connected to the FPC terminal portion 708, and various signals and the like are supplied to the pixel portion 702, the demultiplexer 703, the source driver 704, and the gate driver 706 by the FPC 716. A signal line 710 is connected to the pixel portion 702, the demultiplexer 703, the source driver 704, the gate driver 706, and the FPC terminal portion 708, respectively. Various signals and the like supplied by the FPC 716 are supplied to the pixel portion 702, the demultiplexer 703, the source driver 704, the gate driver 706, and the FPC terminal portion 708 through a signal line 710.

  Further, a plurality of gate drivers 706 may be provided in the display device 700. Although the display device 700 has an example in which the gate driver 706 is formed over the same first substrate 701 as the pixel portion 702 and the source driver 704 is a source driver IC, the present invention is not limited to this structure. For example, the source driver 704 may be formed on the first substrate 701. Note that the source driver IC can be provided by a COG (Chip On Glass) method, a wire bonding method, or the like. Further, the demultiplexer 703 can be omitted.

  Further, the display device 700 can include various elements. Examples of the element include, for example, an electroluminescence (EL) element (an EL element containing an organic substance and an inorganic substance, an organic EL element, an inorganic EL element, an LED, and the like), a light-emitting transistor (a transistor that emits light in response to current), and an electron emission element. Device, liquid crystal device, electronic ink device, electrophoretic device, electrowetting device, plasma display panel (PDP), MEMS (micro electro mechanical system) display (for example, grating light valve (GLV), digital micro mirror device) (DMD), a digital micro shutter (DMS) element, an interferometric modulation (IMOD) element, etc.), a piezoelectric ceramic display, and the like.

  An example of a display device using an EL element includes an EL display. As an example of a display device using an electron-emitting device, there is a field emission display (FED) or an SED type flat display (SED: Surface-electron-emission display). Examples of a display device using a liquid crystal element include a liquid crystal display (a transmissive liquid crystal display, a transflective liquid crystal display, a reflective liquid crystal display, a direct-view liquid crystal display, and a projection liquid crystal display). An example of a display device using an electronic ink element or an electrophoretic element includes electronic paper. Note that when a transflective liquid crystal display or a reflective liquid crystal display is realized, part or all of the pixel electrodes may have a function as a reflective electrode. For example, part or all of the pixel electrode may include aluminum, silver, or the like. Further, in that case, a storage circuit such as an SRAM can be provided below the reflective electrode. Thereby, power consumption can be further reduced.

  Note that as a display method in the display device 700, a progressive method, an interlace method, or the like can be used. In addition, color elements controlled by pixels in color display are not limited to three colors of RGB (R represents red, G represents green, and B represents blue). For example, it may be composed of four pixels of an R pixel, a G pixel, a B pixel, and a W (white) pixel. Alternatively, as in a pen tile arrangement, one color element may be configured by two colors of RGB, and two different colors may be selected and configured according to the color element. Alternatively, one or more colors of yellow, cyan, magenta, and the like may be added to RGB. The size of the display area may be different for each dot of the color element. Note that the disclosed invention is not limited to a color display device, and can be applied to a monochrome display device.

  Further, a coloring layer (also referred to as a color filter) may be used in order to display a display device in full color using white light emission (W) for a backlight (an organic EL element, an inorganic EL element, an LED, a fluorescent lamp, or the like). Good. As the coloring layer, for example, red (R), green (G), blue (B), yellow (Y) and the like can be used in appropriate combination. By using the coloring layer, color reproducibility can be increased as compared with the case where the coloring layer is not used. At this time, by arranging a region having a coloring layer and a region having no coloring layer, white light in a region having no coloring layer may be directly used for display. By arranging a region which does not have a coloring layer in a part, reduction in luminance due to the coloring layer can be reduced in bright display, and power consumption can be reduced by about 20 to 30% in some cases. However, in the case of performing full-color display using a self-luminous element such as an organic EL element or an inorganic EL element, R, G, B, Y, and W may emit light from elements having respective luminescent colors. In some cases, the use of the self-light-emitting element can further reduce power consumption as compared with the case of using a coloring layer.

  As a color conversion method, in addition to a method (color filter method) of converting a part of the light emission from the above-mentioned white light emission into red, green, and blue by passing through a color filter, the light emission of red, green, and blue is also performed. A system (a three-color system) that is used for each, or a system (a color conversion system or a quantum dot system) that converts a part of light emitted from blue light emission into red or green light may be applied.

  In this embodiment, a structure in which a liquid crystal element and an EL element are used as display elements will be described with reference to FIGS. Note that FIG. 22 is a cross-sectional view taken along a dashed-dotted line QR shown in FIG. 21 and has a configuration using a liquid crystal element as a display element. FIG. 23 is a cross-sectional view taken along a dashed-dotted line QR shown in FIG. 21 and has a configuration using an EL element as a display element.

  First, common parts shown in FIGS. 22 and 23 will be described first, and then different parts will be described below.

<Description of common parts of display device>
The display device 700 illustrated in FIGS. 22 and 23 includes a routing wiring portion 711, a pixel portion 702, a demultiplexer 703, and an FPC terminal portion 708. The routing wiring portion 711 has a signal line 710. Further, the pixel portion 702 includes a transistor 750 and a capacitor 790. The demultiplexer 703 includes a transistor 752.

  The transistor 750 and the transistor 752 may be any of a top-gate transistor, a bottom-gate transistor, a channel-etch transistor, and a channel-protection transistor. 22 and 23 illustrate a top gate type. For a semiconductor layer of the transistor 750 and the transistor 752, a silicon semiconductor (amorphous silicon, polycrystalline silicon, or the like), an oxide semiconductor (zinc oxide, indium oxide, or the like), or the like can be used. FIGS. 22 and 23 illustrate the case where an oxide semiconductor is used.

  The capacitor 790 includes a first oxide semiconductor film included in the transistor 750, a lower electrode formed through a step of processing the same oxide semiconductor film, and a conductive element serving as a source electrode and a drain electrode included in the transistor 750. A film and an upper electrode formed through a step of processing the same conductive film. Further, a step of forming the same insulating film between the lower electrode and the upper electrode as an insulating film functioning as a second insulating film of the transistor 750 and an insulating film functioning as a third insulating film is included. There is provided an insulating film that is formed through the process. That is, the capacitor 790 has a stacked structure in which an insulating film functioning as a dielectric is sandwiched between a pair of electrodes.

  In FIGS. 22 and 23, a planarization insulating film 770 is provided over the transistor 750, the transistor 752, and the capacitor 790.

  As the planarization insulating film 770, a heat-resistant organic material such as a polyimide resin, an acrylic resin, a polyimide amide resin, a benzocyclobutene resin, a polyamide resin, or an epoxy resin can be used. Note that the planarization insulating film 770 may be formed by stacking a plurality of insulating films formed using these materials. Further, a structure without the planarization insulating film 770 may be employed.

  Further, the signal line 710 is formed through the same step as the conductive film functioning as the source and drain electrodes of the transistors 750 and 752. Note that the signal line 710 is a conductive film formed through a step different from that of the source and drain electrodes of the transistors 750 and 752, for example, an oxide semiconductor formed through the same step as an oxide semiconductor film functioning as a gate electrode. A membrane may be used. For example, in the case where a material containing a copper element is used for the signal line 710, signal delay or the like due to wiring resistance is small, and display on a large screen is possible.

  The FPC terminal portion 708 includes a connection electrode 760, an anisotropic conductive film 780, and an FPC 716. Note that the connection electrode 760 is formed through the same process as the conductive film functioning as the source and drain electrodes of the transistors 750 and 752. The connection electrode 760 is electrically connected to a terminal included in the FPC 716 through an anisotropic conductive film 780.

  Further, as the first substrate 701 and the second substrate 705, for example, a glass substrate can be used. Alternatively, a flexible substrate may be used as the first substrate 701 and the second substrate 705. Examples of the flexible substrate include a plastic substrate and the like.

  Further, a structure 778 is provided between the first substrate 701 and the second substrate 705. The structure 778 is a columnar spacer obtained by selectively etching an insulating film, and is provided to control a distance (cell gap) between the first substrate 701 and the second substrate 705. Note that a spherical spacer may be used as the structure 778.

  Further, a light-blocking film 738 functioning as a black matrix, a coloring film 736 functioning as a color filter, and an insulating film 734 in contact with the light-blocking film 738 and the coloring film 736 are provided on the second substrate 705 side.

<Example of configuration of display device using liquid crystal element>
The display device 700 illustrated in FIG. 22 includes a liquid crystal element 775. The liquid crystal element 775 includes a conductive film 772, a conductive film 774, and a liquid crystal layer 776. The conductive film 774 is provided on the second substrate 705 side and has a function as a counter electrode. The display device 700 illustrated in FIG. 22 can display an image by controlling transmission and non-transmission of light by changing the alignment state of the liquid crystal layer 776 by a voltage applied to the conductive films 772 and 774.

  The conductive film 772 is connected to a conductive film of the transistor 750 that functions as a source electrode and a drain electrode. The conductive film 772 is formed over the planarization insulating film 770 and functions as a pixel electrode, that is, one electrode of a display element. The conductive film 772 has a function as a transparent electrode. A display device 700 illustrated in FIG. 22 is a so-called transmission type color liquid crystal display device in which light from a backlight is transmitted through a liquid crystal layer 776 and displayed through a coloring film 736.

  As the conductive film 772, a conductive film having a property of transmitting visible light or a conductive film having a property of reflecting visible light can be used. As the conductive film which transmits visible light, for example, a material containing one kind selected from indium (In), zinc (Zn), and tin (Sn) may be used. As the conductive film which reflects visible light, for example, a material containing aluminum or silver may be used.

  Although not illustrated in FIG. 22, an alignment film may be provided on each of the conductive films 772 and 774 on the side in contact with the liquid crystal layer 776. Although not shown in FIG. 22, optical members (optical substrates) such as a polarizing member, a retardation member, and an anti-reflection member may be appropriately provided. For example, circularly polarized light from a polarizing substrate and a phase difference substrate may be used. Further, a backlight, a sidelight, or the like may be used as a light source.

  When a liquid crystal element is used as a display element, a thermotropic liquid crystal, a low-molecular liquid crystal, a polymer liquid crystal, a polymer dispersed liquid crystal, a ferroelectric liquid crystal, an antiferroelectric liquid crystal, or the like can be used. These liquid crystal materials exhibit a cholesteric phase, a smectic phase, a cubic phase, a chiral nematic phase, an isotropic phase, or the like depending on conditions.

  In the case where the horizontal electric field method is employed, a liquid crystal exhibiting a blue phase without using an alignment film may be used. The blue phase is one of the liquid crystal phases, and is a phase that appears when the temperature of the cholesteric liquid crystal is increased immediately before the transition from the cholesteric phase to the isotropic phase. Since the blue phase appears only in a narrow temperature range, a liquid crystal composition in which several percent by weight or more of a chiral agent is mixed is used for the liquid crystal layer in order to improve the temperature range. A liquid crystal composition containing a liquid crystal exhibiting a blue phase and a chiral agent has a short response speed and is optically isotropic, and thus does not require an alignment treatment. Further, since it is not necessary to provide an alignment film, rubbing treatment is not required, so that electrostatic breakdown caused by the rubbing treatment can be prevented, and defects and breakage of the liquid crystal display device during the manufacturing process can be reduced. . Further, a liquid crystal material exhibiting a blue phase has a small viewing angle dependence.

  In the case where a liquid crystal element is used as a display element, a TN (Twisted Nematic) mode, an IPS (In-Plane-Switching) mode, an FFS (Fringe Field Switching) mode, an ASM (Axially Symmetrical Microcellular, O-cell, O-cell). A Compensated Birefringence (FLC) mode, an FLC (Ferroelectric Liquid Crystal) mode, an AFLC (Anti Ferroelectric Liquid Crystal) mode, or the like can be used.

  Further, a normally black liquid crystal display device, for example, a transmissive liquid crystal display device employing a vertical alignment (VA) mode may be used. There are several examples of the vertical alignment mode. For example, an MVA (Multi-Domain Vertical Alignment) mode, a PVA (Patterned Vertical Alignment) mode, an ASV mode, or the like can be used.

<Display using light emitting element>
The display device 700 illustrated in FIG. 23 includes a light-emitting element 782. The light-emitting element 782 includes a conductive film 784, an EL layer 786, and a conductive film 788. The display device 700 illustrated in FIG. 23 can display an image when the EL layer 786 included in the light-emitting element 782 emits light.

  The conductive film 784 is connected to a conductive film of the transistor 750 that functions as a source electrode and a drain electrode. The conductive film 784 is formed over the planarization insulating film 770 and functions as a pixel electrode, that is, one electrode of a display element. As the conductive film 784, a conductive film having a property of transmitting visible light or a conductive film having a property of reflecting visible light can be used. As the conductive film which transmits visible light, for example, a material containing one kind selected from indium (In), zinc (Zn), and tin (Sn) may be used. As the conductive film which reflects visible light, for example, a material containing aluminum or silver may be used.

  In the display device 700 illustrated in FIG. 23, an insulating film 730 is provided over the planarizing insulating film 770 and the conductive film 784. The insulating film 730 covers part of the conductive film 784. Note that the light-emitting element 782 has a top emission structure. Therefore, the conductive film 788 has a light-transmitting property and transmits light emitted from the EL layer 786. In the present embodiment, the top emission structure is exemplified, but the invention is not limited to this. For example, the present invention can be applied to a bottom emission structure in which light is emitted to the conductive film 784 side and a dual emission structure in which light is emitted to both the conductive film 784 and the conductive film 788.

  Further, a coloring film 736 is provided at a position overlapping with the light-emitting element 782, and a light-blocking film 738 is provided at a position overlapping with the insulating film 730, the wiring portion 711, and the source driver 704. The coloring film 736 and the light-blocking film 738 are covered with an insulating film 734. The space between the light-emitting element 782 and the insulating film 734 is filled with a sealing film 732. Note that in the display device 700 illustrated in FIG. 23, the structure in which the coloring film 736 is provided is illustrated; however, the present invention is not limited to this. For example, in the case where the EL layers 786 are separately formed, a structure in which the coloring film 736 is not provided may be employed.

  The structure described in this embodiment can be used in appropriate combination with any of the structures described in the other embodiments.

(Embodiment 4)
In this embodiment, application examples of the display device described in the above embodiment to an automobile and other moving objects are described below with reference to FIGS.

<Example of application of display system to moving objects>
An example in which the display device included in the above-described display system is provided in the vicinity of a driver's seat of a vehicle, which is a moving body, will be described.

  For example, FIG. 24A shows the interior of an automobile in which bench seats are used for a driver seat and a passenger seat. FIG. 24A illustrates a display device 52A provided on a door portion, a display device 52B provided on a handle, and a display device 52C provided on a central portion of a seat surface of a bench seat.

  For example, the display device 52A can complement the view blocked by the door by displaying an image from an imaging device provided on the vehicle body on the display unit.

  The display devices 52B and 52C include various other information such as navigation information, meters such as a speedometer and a tachometer, a mileage, a refueling amount, a gear state, an air conditioner setting, in addition to an image from an imaging device provided on the vehicle body. Can be provided. Further, display items, layout, and the like displayed on the display device can be appropriately changed according to the user's preference. The display devices 52B and 52C can also be used as lighting devices.

  FIG. 24B is a diagram illustrating the vicinity of a windshield in a vehicle. FIG. 24B illustrates a display device 53A attached to a dashboard.

  The display device 53A can provide various other information such as navigation information, speedometer and tachometer, mileage, refueling amount, gear state, setting of air conditioner, and the like. Further, display items, layouts, and the like displayed on the display device can be appropriately changed according to the user's preference, so that design can be improved. The display device 53A can be used as a lighting device.

  The display device 53A can complement the field of view (blind spot) obstructed by the vehicle body by displaying an image from the imaging means provided on the vehicle body. That is, by displaying an image from the image pickup means provided outside the automobile, blind spots can be compensated for and safety can be improved. In addition, by displaying an image that complements the invisible part, it is possible to more naturally confirm safety without a sense of incongruity. The display device 53A can be used as a lighting device.

<Example of moving object>
An example of a moving object will be described.

  The display system according to one embodiment of the present invention can be used for not only automobiles but also various moving objects. Specific examples of these moving objects are shown in FIGS.

  FIG. 25A illustrates a bus 302. The moving object according to one embodiment of the present invention can be used for the bus 302. The display system can capture an image outside the bus 302 and improve the visibility of the image when the image is visually recognized. Therefore, the bus 302 with improved safety can be provided.

  FIG. 25B illustrates a train 303. The moving object according to one embodiment of the present invention can be used for the train 303. The display system can capture an image outside the train 303 and can enhance the visibility of the image when the image is visually recognized. Therefore, a train 303 with improved safety can be provided.

  FIG. 25C illustrates an airplane 304. The moving object according to one embodiment of the present invention can be used for an airplane 304. The display system captures an image outside the airplane 304 and can enhance the visibility of the image when visually recognizing the image. Therefore, the airplane 304 can have higher safety.

<Supplementary notes regarding the description in this specification etc.>
In this specification and the like, the ordinal numbers “first”, “second”, and “third” are given in order to avoid confusion between components. Therefore, the number of components is not limited. In addition, the order of the components is not limited.

  In this specification and the like, in the block diagrams, components are classified according to functions and are shown as blocks independent of each other. However, in an actual circuit or the like, it is difficult to separate constituent elements for each function, and a plurality of functions may be involved in one circuit, or one function may be involved in a plurality of circuits. Therefore, the blocks in the block diagram are not limited to the components described in the specification, and can be appropriately paraphrased according to the situation.

  In the drawings, the same element or an element having a similar function, an element of the same material, an element formed at the same time, or the like may be denoted by the same reference numeral, and a repeated description thereof may be omitted.

  In this specification and the like, the connection relation between transistors is described as “one of a source or a drain” (or a first electrode or a first terminal), “the other of a source or a drain” (or a second electrode, or a second electrode). Terminal). This is because the source and the drain of the transistor change depending on the structure, operating conditions, and the like of the transistor. Note that the terms “source” and “drain” of a transistor can be appropriately reworded depending on the situation, such as a source (drain) terminal and a source (drain) electrode.

  In this specification and the like, voltage and potential can be paraphrased as appropriate. The voltage refers to a potential difference from a reference potential. For example, if the reference potential is a ground potential (ground potential), the voltage can be rephrased to a potential. The ground potential does not always mean 0V. Note that the potential is relative, and the potential given to a wiring or the like may be changed depending on a reference potential.

  In this specification and the like, a switch is a switch which is turned on or off and has a function of controlling whether a current flows or not. Alternatively, a switch has a function of selecting and switching a path through which a current flows.

  As an example, an electric switch, a mechanical switch, or the like can be used. That is, the switch is not limited to a specific switch as long as it can control the current.

  Note that in the case where a transistor is used as a switch, the “conduction state” of the transistor refers to a state in which the source and the drain of the transistor can be regarded as being electrically short-circuited. The “non-conductive state” of a transistor refers to a state in which the source and the drain of the transistor can be regarded as being electrically disconnected. Note that when the transistor is operated as a simple switch, the polarity (conductivity type) of the transistor is not particularly limited.

  In this specification and the like, the expression "A and B are connected" includes a case where A and B are directly connected and a case where A and B are electrically connected. Here, "A and B are electrically connected" means that when there is an object having some kind of electrical action between A and B, it is possible to exchange electric signals between A and B. To say.

  10: display system, 10A: display system, 10B: display system, 10C: display system, 11: imaging device, 11A: imaging device, 12: feature amount output circuit, 13: database, 14: image processing circuit, 15: display Apparatus, 16: imaging data, 17: feature data, 18: correction data, 19: detection data, 20: transmission / reception circuit, 21: network, 22: automobile, 22A: automobile, 23: data processing circuit, 24: Relay station, 25: sensor, S11-S17: step, S21-S26: step, 30: neural network, 31: input layer, 32: intermediate layer, 33: output layer, 34: filter, 35: convolutional data, 36: Pooling data, 37: filter, 38: convolutional data, 39: pooling data, 40: all combined data, 41 Output data, 42: automobile, 43: light source, 44: irradiation direction, 45: imaging direction, 46: lens, 47: water drop, 48: sharp area, 49: unsharp area, 11_1-11_n: imaging apparatus, 15_1-15_n: Display device, S31-S34: Step, 18A: Image, 19A: Image, 17A: Feature data, 51: Image data, 700: Display device, 701: Substrate, 702: Pixel unit, 703: Demultiplexer, 704: Source Driver, 705: substrate, 706: gate driver, 708: FPC terminal portion, 710: signal line, 711: wiring portion, 712: sealing material, 716: FPC, 730: insulating film, 732: sealing film, 734: insulating Film, 736: coloring film, 738: light shielding film, 750: transistor, 752: transistor, 760: connection electrode, 770: flattening Edge film, 772: conductive film, 774: conductive film, 775: liquid crystal element, 776: liquid crystal layer, 778: structure, 780: anisotropic conductive film, 782: light emitting element, 784: conductive film, 786: EL layer 788: conductive film, 790: capacitor, 52A-52C: display device, 53A: display device, 302: bus, 303: train, 304: airplane

Claims (6)

An imaging device, a display device, a feature amount output circuit, an image processing circuit, and a database,
The imaging device has a function of outputting imaging data,
The feature amount output circuit has a function of outputting feature amount data of the imaging data,
The database has correction data and detection data, and has a function of outputting the correction data to the image processing circuit according to the feature amount data,
The image processing circuit has a function of generating image data by correcting the imaging data based on the correction data,
The display system according to claim 1, wherein the display device has a function of performing display according to the image data. In claim 1,
The display system according to claim 1, wherein the database has a function of selecting the detection data that matches or is similar to the feature amount data and outputting the correction data corresponding to the detection data to the image processing circuit. In claim 1,
The database has a function of updating the detection data serving as a weight parameter by machine learning using the feature data as learning data, and inferring the correction data that matches or is similar to the feature data. A display system comprising: An imaging device, a display device, a feature amount output circuit, an image processing circuit, and a transmission / reception circuit,
The imaging device has a function of acquiring imaging data,
The feature amount output circuit has a function of acquiring feature amount data of the imaging data,
The feature amount data has a function of being transmitted to a database having correction data and detection data via a transmission / reception circuit,
The image processing circuit has a function of receiving the correction data from the database via the transmission / reception circuit and generating image data by correcting the imaging data based on the correction data,
The moving object, wherein the display device has a function of performing display according to the image data. In claim 4,
The correction data is data corresponding to the detection data,
The moving object, wherein the detection data is data that matches or is similar to the feature amount data. In claim 4,
The correction data is data obtained by inputting the feature data in the database in which the detection data serving as a weight parameter is updated by machine learning using the feature data as learning data. A mobile object characterized by the fact that:

Images