JP2010062940A - Image display/image detection apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To detect that a casing has been folded when reading an image for detection displayed on one of two inner surfaces opposing while the casing has been folded on the other inner surface. <P>SOLUTION: An image display/image detection apparatus having the casing that is folded in two includes: an image storage section 24 for detection for correlating a prescribed image for detection with prescribed processing for storage; an image display processing section 124 for detection for displaying the image for detection on a liquid crystal panel with a built-in sensor at one display/photosensor section 300 while the casing has not been folded; and an image scan section 126 for detection for executing the prescribed processing correlated with the image for detection when the image for detection, which has been displayed on the liquid crystal panel with a built-in sensor of the other display/photosensor section 300 by the image display processing section 124 for detection, is detected while the casing has been folded. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image display / image detection apparatus capable of simultaneously displaying an image and reading an object, and particularly capable of reading an object such as a document by a simple operation.

  2. Description of the Related Art Conventionally, scanner devices that can read information described in documents and digitize it as image data have been widely used. In recent years, portable scanner devices have appeared. As an example, there are a mobile phone with a camera capable of reading characters and the like, a handy scanner provided in a facsimile machine, and the like.

  In addition, in order to improve user convenience, a scanner device that can simultaneously read both sides of a document with a single operation has been used. Such a scanner device is generally large and stationary, such as a copier, but a portable type has also appeared (see Patent Document 1).

  The scanner device disclosed in Patent Document 1 has a structure in which two scanner units are folded at a hinge portion, and can simultaneously read both sides of a document sandwiched between the folded scanner units.

Patent Document 2 discloses an image information processing apparatus that acquires the reading result of both sides of a document by a scanner device and displays the result on a display unit of the device itself.
JP 2008-66961 A (publication date: March 21, 2008) JP-A-8-88741 (Publication date: April 2, 1996)

  The scanner device disclosed in Patent Document 1 is provided with a mechanical switch for detecting folding or opening / closing of a scanner unit in a hinge portion or the like, and the scanner is triggered when the switch is turned on. Start reading objects such as documents sandwiched between units. Further, in a conventional scanner device such as a copying machine, reading of an object such as a document is generally started when a user presses a button for instructing reading.

  However, when reading is triggered by the switch being turned on, for example, when the user simply folds the scanner unit without aiming to read an object such as a document. However, reading may be started.

  In addition, when reading is started as a trigger when the user presses a button or the like instructing reading, reading may be started even when the user does not fold the scanner unit.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide an image display / image detection having a housing that can be folded in two and can simultaneously perform image display and reading of an object. When the detection image displayed on one of the two inner surfaces facing each other in the folded state of the casing can be read on the other inner surface, the casing is folded. It is an object of the present invention to provide an image display / image detection apparatus that can detect the presence of the image.

  In order to solve the above problems, an image display / image detection apparatus according to the present invention has a housing that can be folded in two, and two inner surfaces facing each other in the folded state of the housing have one The image display / image detection apparatus has a first planar member for displaying an image on the inner surface and a second planar member for detecting a nearby image on the other inner surface so as to face each other. A detection image storage unit that stores a predetermined detection image and information indicating a predetermined process that can be executed by the apparatus, and the detection image is displayed on the first planar member. The detection image display instruction means and the detection image displayed on the first planar member by the detection image display instruction means are detected by the second planar member in a state where the casing is folded. Shows the predetermined processing associated with the detection image. The broadcast is acquired from the detecting image storage unit, and characterized by comprising a process execution means for executing the predetermined process shown in the acquired information.

  According to the above configuration, it is possible to store a predetermined detection image and information indicating a predetermined process executed by the own apparatus in association with each other. The detection image can be displayed on the first planar member. When the detection image displayed on the first planar member by the detection image display instruction means is detected by the second planar member in a state where the casing is folded, the detection is performed. Information indicating the predetermined process associated with the image for use can be acquired from the detection image storage unit, and the predetermined process indicated by the acquired information can be executed.

  Therefore, the user can fold the casing in a state where the detection image is displayed on the first planar member, and the detection image can be detected by the second planar member. The predetermined process corresponding to the detection image can be executed.

  Accordingly, even if a mechanical switch that is generally provided in a hinge portion of an apparatus having a so-called clamshell type housing and can detect opening and closing of the housing is not provided, the user can operate the housing. By folding the body, it is possible to execute the predetermined processing corresponding to the detection image. Note that when the detection image displayed on the first planar member is detected by the second planar member, it can be seen that the casing is in a folded state.

  Further, in the mechanism in which the mechanical switch detects the opening / closing of the housing and executes the predetermined processing, the user executes the predetermined processing every time the user accidentally opens / closes the housing. In the image display / image detection apparatus according to the present invention, even if the user accidentally folds the housing, the first image is not displayed on the first planar member. Since the detection image is not detected by the two-plane member, it is possible to prevent the predetermined process from being erroneously executed.

  In addition, since the predetermined processing is associated with the detection image, the user simply folds the casing, and the predetermined processing according to the detection image displayed on the first planar member is performed. It is possible to execute the process.

  Furthermore, in the image display / image detection apparatus according to the present invention, in the above configuration, the predetermined processing is placed on the first planar member or the second planar member in a state where the casing is folded. It is good also as a structure which is the process which detects the image of the made target object with the said 2nd planar member.

  According to the above configuration, the predetermined processing is further placed on the first planar member or the second planar member by the second planar member with the casing folded. This is a process for detecting the image of the target object.

  Therefore, when the user folds the casing in a state where the detection image is displayed on the first planar member, the first planar member or the second surface is obtained by the second planar member. The process which detects the image of the target object mounted on the shaped member can be executed.

  Therefore, even if it does not depend on a mechanical switch, the effect that the process which detects the image of the side facing the said 2nd planar member of a target object can be performed by a user folding the said housing | casing. Play.

  Further, in the image display / image detection apparatus according to the present invention, even if the user accidentally folds the casing, the first image is not displayed on the first planar member unless the detection image is displayed. Since the detection image is not detected by the two-plane member, there is an effect that the process of detecting the image of the object can be prevented from being erroneously executed.

  Furthermore, in the image display / image detection apparatus according to the present invention, in the configuration described above, the first planar member further detects a nearby image, and the predetermined processing is performed by folding the casing. It is good also as a process which is the process which detects the image of the said target object mounted in the said 1st planar member or the said 2nd planar member in the said state with the said 1st planar member.

  According to the above configuration, the first planar member can detect a nearby image, and the predetermined processing is performed by the first planar member in a state where the casing is folded. This is processing for detecting an image of the object.

  Therefore, when the user folds the casing in a state where the detection image is displayed on the first planar member, the first planar member or the second surface is obtained by the first planar member. The process which detects the image of the target object mounted on the shaped member can be executed.

  Therefore, even if it does not depend on a mechanical switch, the effect that the process which detects the image of the side which faces the said 1st planar member of a target object can be performed by a user folding up the above-mentioned case. Play.

  Further, in the image display / image detection apparatus according to the present invention, in the above configuration, the detection image display instruction unit receives a user instruction to display the detection image, and in response to the user instruction, the first display The detection image may be displayed on a single planar member.

  According to said structure, while receiving the user instruction | indication which displays the said image for a detection, according to the said user instruction | indication, the said image for a detection can be displayed on the said 1st planar member.

  Therefore, the detection image can be displayed on the first planar member, triggered by a user instruction.

  Therefore, after the detection image is displayed on the first planar member according to a user instruction, the user folds the casing, so that at least one of the first planar member and the second planar member is used. There is an effect that it is possible to execute a process of detecting an image of an object placed on the first planar member or the second planar member.

  On the contrary, even if the user folds the casing without giving a user instruction to display the detection image on the first planar member, the first planar member or the second planar member is placed. The process of detecting the image of the target object is not executed. Therefore, even when the user does not intend to detect the image of the object but simply folds the housing, it is possible to prevent the process of detecting the image of the object from being erroneously executed. There is an effect.

  Furthermore, the image display / image detection apparatus according to the present invention may be configured such that, in the above configuration, the predetermined process is a process of turning off the power of the own apparatus.

  According to the above configuration, the predetermined process is a process of turning off the power of the own apparatus.

  Therefore, the power of the image display / image detection apparatus can be turned off by the user folding the casing while the detection image is displayed on the first planar member.

  Therefore, there is an effect that the power of the image display / image detection apparatus can be turned off when the user folds the casing without depending on the mechanical switch.

  Furthermore, the image display / image detection apparatus according to the present invention may be configured such that, in the above configuration, the detection image display instruction means displays the detection image in a region where the object is placed.

  According to said structure, the said image for a detection can be further displayed on the area | region which mounts a target object.

  Therefore, in a state where the detection image is displayed in the region on which the object on the first planar member is placed, the user folds the casing so that the detection is performed on the second planar member. Since the image is detected, the predetermined processing corresponding to the detection image can be executed.

  Therefore, in particular, when the predetermined process is a process of turning off the power of the own apparatus, when the user folds the casing and the detection image is detected by the second planar member, You can turn off your device. In other words, when the user accidentally folds the housing without placing the object in the area where the object is placed, the user does not perform the process of detecting the image of the object, There is an effect that it can be turned off.

  As described above, the image display / image detection device according to the present invention has a housing that can be folded in two, and the two inner surfaces that face each other in the folded state of the housing are arranged on one inner surface, An image display / image detection device in which a first planar member for displaying an image is disposed on the other inner surface so that a second planar member for detecting a nearby image is opposed to each other. A detection image storage unit for storing the detection image and information indicating a predetermined process executable by the apparatus in association with each other, and a detection image for displaying the detection image on the first planar member When the detection image display instruction means detects the detection image displayed on the first planar member by the second planar member with the display instruction means and the housing folded, Information indicating the predetermined process associated with the detection image is Obtained from known image storage unit, and a processing executing means for executing the predetermined process shown in the acquired information.

  Therefore, the user can fold the casing in a state where the detection image is displayed on the first planar member, and the detection image can be detected by the second planar member. The predetermined process corresponding to the detection image can be executed.

  Therefore, the user can fold the housing without relying on a mechanical switch that is generally provided in a hinge portion of an apparatus having a so-called clamshell housing and can detect opening and closing of the housing. As a result, the predetermined processing corresponding to the detection image can be executed.

  The data display / sensor device 100 (image display / image detection device) according to the present embodiment has a housing that can be folded in two, and each of the two inner surfaces facing each other in the folded state of the housing, Sensor built-in liquid crystal panels 301 (first planar member, second planar member) are arranged so as to face each other, and the light contained in each sensor built-in liquid crystal panel 301 in a state where the casing is folded. By driving the sensor circuit 32, at least one of the front and back surfaces of an object such as a document sandwiched between two sensor-equipped liquid crystal panels 301 is scanned, and image data obtained as a result of scanning is scanned. This is a device capable of displaying on at least one of the two sensor-equipped liquid crystal panels 301. Then, the data display / sensor device 100 displays the detection image on one sensor built-in liquid crystal panel 301, and can read the detection image as a result of scanning with the other sensor built-in liquid crystal panel 301. Processing associated with the detection image (for example, scanning of the target object, power-off of the own device, etc.) is executed.

  First, an outline of the sensor built-in liquid crystal panel 301 included in the data display / sensor device 100 will be described below.

(Outline of LCD panel with built-in sensor)
The sensor built-in liquid crystal panel 301 provided in the data display / sensor device 100 is a liquid crystal panel capable of detecting an image of an object in addition to displaying data. Here, the image detection of the object is, for example, detection of a position pointed (touched) by the user with a finger or a pen, or reading (scanning) of an image of a printed material. The device used for display is not limited to a liquid crystal panel, and may be an organic EL (Electro Luminescence) panel or the like.

  The structure of the sensor built-in liquid crystal panel 301 will be described with reference to FIG. FIG. 2 is a diagram schematically showing a cross section of the sensor built-in liquid crystal panel 301. The sensor built-in liquid crystal panel 301 described here is an example, and any structure can be used as long as the display surface and the reading surface are shared.

  As shown in the figure, the sensor built-in liquid crystal panel 301 includes an active matrix substrate 51A disposed on the back surface side and a counter substrate 51B disposed on the front surface side, and has a structure in which a liquid crystal layer 52 is sandwiched between these substrates. is doing. The active matrix substrate 51A is provided with a pixel electrode 56, a data signal line 57, an optical sensor circuit 32 (not shown), an alignment film 58, a polarizing plate 59, and the like. The counter substrate 51B is provided with color filters 53r (red), 53g (green), 53b (blue), a light shielding film 54, a counter electrode 55, an alignment film 58, a polarizing plate 59, and the like. In addition, a backlight 307 is provided on the back surface of the sensor built-in liquid crystal panel 301.

  The photodiode 6 included in the photosensor circuit 32 is provided in the vicinity of the pixel electrode 56 provided with the blue color filter 53b, but is not limited to this configuration. It may be provided in the vicinity of the pixel electrode 56 provided with the red color filter 53r, or may be provided in the vicinity of the pixel electrode 56 provided with the green color filter 53g.

  Next, with reference to FIGS. 3A and 3B, two types of methods for detecting the position where the user touches the sensor built-in liquid crystal panel 301 with a finger or a pen will be described.

  FIG. 3A is a schematic diagram illustrating a state in which a position touched by the user is detected by detecting a reflected image. When the light 63 is emitted from the backlight 307, the optical sensor circuit 32 including the photodiode 6 detects the light 63 reflected by the object 64 such as a finger. Thereby, the reflected image of the target object 64 can be detected. Thus, the sensor built-in liquid crystal panel 301 can detect the touched position by detecting the reflected image.

  FIG. 3B is a schematic diagram illustrating a state in which a position touched by the user is detected by detecting a shadow image. As shown in FIG. 3B, the optical sensor circuit 32 including the photodiode 6 detects external light 61 transmitted through the counter substrate 51B and the like. However, when there is an object 62 such as a pen, the incident of the external light 61 is hindered, so that the amount of light detected by the optical sensor circuit 32 is reduced. Thereby, a shadow image of the object 62 can be detected. Thus, the sensor built-in liquid crystal panel 301 can also detect a touched position by detecting a shadow image.

  As described above, the photodiode 6 may detect reflected light (shadow image) of the light emitted from the backlight 307 or may detect a shadow image caused by external light. Further, the two types of detection methods may be used in combination to detect both a shadow image and a reflected image at the same time.

(Data display / sensor configuration)
Next, with reference to FIG. 4, the configuration of the main part of the data display / sensor device 100 will be described. FIG. 4 is a block diagram showing a main configuration of the data display / sensor device 100. As shown in FIG. As illustrated, the data display / sensor device 100 includes one or more display / light sensor units 300, a circuit control unit 600, a data processing unit 700, a main control unit 800, a storage unit 901, a primary storage unit 902, and an operation unit. 903, an external communication unit 907, an audio output unit 908, and an audio input unit 909. Here, the data display / sensor device 100 will be described as including two display / light sensor units 300 (first display / light sensor unit 300A and second display / light sensor unit 300B). When the first display / light sensor unit 300A and the second display / light sensor unit 300B are not distinguished, they are referred to as the display / light sensor unit 300.

  The display / light sensor unit 300 is a so-called liquid crystal display device with a built-in light sensor. The display / light sensor unit 300 includes a sensor built-in liquid crystal panel 301, a backlight 307, and a peripheral circuit 309 for driving them.

  The sensor built-in liquid crystal panel 301 includes a plurality of pixel circuits 31 and photosensor circuits 32 arranged in a matrix. The detailed configuration of the sensor built-in liquid crystal panel 301 will be described later.

  The peripheral circuit 309 includes a liquid crystal panel drive circuit 304, an optical sensor drive circuit 305, a signal conversion circuit 306, and a backlight drive circuit 308.

  The liquid crystal panel driving circuit 304 outputs a control signal (G) and a data signal (S) in accordance with the timing control signal (TC1) and the data signal (D) from the display control unit 601 of the circuit control unit 600, and the pixel circuit 31. It is a circuit which drives. Details of the driving method of the pixel circuit 31 will be described later.

  The optical sensor driving circuit 305 is a circuit that drives the optical sensor circuit 32 by applying a voltage to the signal line (R) in accordance with a timing control signal (TC2) from the sensor control unit 602 of the circuit control unit 600. Details of the driving method of the optical sensor circuit 32 will be described later.

  The signal conversion circuit 306 is a circuit that converts the sensor output signal (SS) output from the optical sensor circuit 32 into a digital signal (DS) and transmits the converted signal to the sensor control unit 602.

  The backlight 307 includes a plurality of white LEDs (Light Emitting Diodes) and is disposed on the back surface of the sensor built-in liquid crystal panel 301. When a power supply voltage is applied from the backlight drive circuit 308, the backlight 307 is turned on and irradiates the sensor built-in liquid crystal panel 301 with light. Note that the backlight 307 is not limited to white LEDs, and may include LEDs of other colors. The backlight 307 may include, for example, a cold cathode fluorescent lamp (CCFL) instead of the LED.

  The backlight driving circuit 308 applies a power supply voltage to the backlight 307 when the control signal (BK) from the backlight control unit 603 of the circuit control unit 600 is at a high level, and conversely, the backlight control unit 603. When the control signal from is at a low level, no power supply voltage is applied to the backlight 307.

  Next, the circuit control unit 600 will be described. The circuit control unit 600 has a function as a device driver that controls the peripheral circuit 309 of the display / light sensor unit 300. The circuit control unit 600 includes a display control unit 601, a sensor control unit 602, a backlight control unit 603, and a display data storage unit 604.

  The display control unit 601 receives display data from the display data processing unit 701 of the data processing unit 700, and performs timing control on the liquid crystal panel driving circuit 304 of the display / light sensor unit 300 in accordance with an instruction from the display data processing unit 701. A signal (TC1) and a data signal (D) are transmitted, and the received display data is displayed on the sensor built-in liquid crystal panel 301.

  The display control unit 601 temporarily stores the display data received from the display data processing unit 701 in the display data storage unit 604. Then, a data signal (D) is generated based on the primary stored display data. The display data storage unit 604 is, for example, a video random access memory (VRAM).

  The sensor control unit 602 transmits a timing control signal (TC2) to the optical sensor driving circuit 305 of the display / optical sensor unit 300 in accordance with an instruction from the sensor data processing unit 703 of the data processing unit 700, and the sensor built-in liquid crystal panel 301. Run the scan with.

  In addition, the sensor control unit 602 receives a digital signal (DS) from the signal conversion circuit 306. Then, image data is generated based on the digital signal (DS) corresponding to the sensor output signal (SS) output from all the optical sensor circuits 32 included in the sensor built-in liquid crystal panel 301. That is, the image data read in the entire reading area of the sensor built-in liquid crystal panel 301 is generated. Then, the generated image data is transmitted to the sensor data processing unit 703.

  The backlight control unit 603 transmits a control signal (BK) to the backlight drive circuit 308 of the display / light sensor unit 300 in accordance with instructions from the display data processing unit 701 and the sensor data processing unit 703 to drive the backlight 307. Let

  When the data display / sensor device 100 includes a plurality of display / light sensor units 300, the display control unit 601 determines which display / light sensor unit 300 displays the display data from the data processing unit 700. When an instruction is received, the liquid crystal panel drive circuit 304 of the display / light sensor unit 300 is controlled according to the instruction. When the sensor control unit 602 receives an instruction from the data processing unit 700 as to which display / light sensor unit 300 is to scan the object, The sensor drive circuit 305 is controlled and a digital signal (DS) is received from the signal conversion circuit 306 of the display / light sensor unit 300 according to the instruction.

  Next, the data processing unit 700 will be described. The data processing unit 700 has a function as middleware that gives an instruction to the circuit control unit 600 based on a “command” received from the main control unit 800. Details of the command will be described later.

  The data processing unit 700 includes a display data processing unit 701 and a sensor data processing unit 703. When the data processing unit 700 receives a command from the main control unit 800, at least one of the display data processing unit 701 and the sensor data processing unit 703 depends on the value of each field (described later) included in the received command. One works.

  The display data processing unit 701 receives display data from the main control unit 800, and gives instructions to the display control unit 601 and the backlight control unit 603 according to the command received by the data processing unit 700, and displays the received display data. The image is displayed on the sensor built-in liquid crystal panel 301. The operation of the display data processing unit 701 according to the command will be described later.

  The sensor data processing unit 703 gives an instruction to the sensor control unit 602 and the backlight control unit 603 according to the command received by the data processing unit 700.

  The sensor data processing unit 703 receives image data from the sensor control unit 602 and stores the image data in the image data buffer 704 as it is. Then, in accordance with the command received by the data processing unit 700, the sensor data processing unit 703 performs “whole image data”, “partial image data (partial image coordinate data) based on the image data stored in the image data buffer 704. At least one of “including coordinate data” and “coordinate data” is transmitted to the main control unit 800. The whole image data, partial image data, and coordinate data will be described later. The operation of the sensor data processing unit 703 according to the command will be described later.

  Next, the main control unit 800 executes an application program. The main control unit 800 reads the program stored in the storage unit 901 into a primary storage unit 902 configured by, for example, a RAM (Random Access Memory) and executes the program.

  An application program executed by the main control unit 800 causes the data processing unit 700 to display display data on the sensor built-in liquid crystal panel 301 or to scan an object on the sensor built-in liquid crystal panel 301. Send commands and display data. When “data type” is designated as a command, at least one of whole image data, partial image data, and coordinate data is received from the data processing unit 700 as a response to the command.

  The circuit control unit 600, the data processing unit 700, and the main control unit 800 can be configured by a CPU (Central Processing Unit), a memory, and the like, respectively. The data processing unit 700 may be configured by a circuit such as an ASIC (application specific integrate circuit).

  Next, the storage unit 901 stores programs and data executed by the main control unit 800 as shown in the figure. The program executed by the main control unit 800 may be separated into an application-specific program and a general-purpose program that can be shared by each application.

  Next, the operation unit 903 receives an input operation of the user of the data display / sensor device 100. The operation unit 903 includes input devices such as a switch, a remote controller, a mouse, and a keyboard, for example. Then, the operation unit 903 generates a control signal corresponding to the user's input operation of the data display / sensor device 100, and transmits the generated control signal to the main control unit 800.

  As an example of the switch, a hinge switch 904 that is provided at the hinge portion of the housing and detects the open / closed state of the housing, a power switch 905 that switches power on and off, and a predetermined function are assigned in advance. A hardware switch such as a user switch 906 is assumed.

  In addition, the data display / sensor device 100 includes an external communication unit 907 for communicating with an external device by wireless / wired communication, an audio output unit 908 such as a speaker for outputting audio, and a microphone for inputting an audio signal. A voice input unit 909 such as the above may be provided as appropriate.

(Command details)
Next, details of commands transmitted from the main control unit 800 to the data processing unit 700 will be described with reference to FIGS. 5 and 6. FIG. 5 is a diagram schematically illustrating an example of a command frame structure. FIG. 6 is a diagram for explaining an example of values that can be specified for each field included in the command and an outline thereof.

  As shown in FIG. 5, the commands are “header”, “data acquisition timing”, “data type”, “scan method”, “scan image gradation”, “scan resolution”, “scan panel”, “display panel”. "And" Reserve "fields. In each field, for example, values shown in FIG. 6 can be designated.

  The “header” field is a field indicating the start of a frame. As long as it is possible to identify the “header” field, the value of the “header” field may be any value.

  Next, the “data acquisition timing” field is a field for designating a timing at which data should be transmitted to the main control unit 800. In the “data acquisition timing” field, for example, values “00” (sense), “01” (event), and “10” (all) can be specified.

  Here, “sense” specifies that the latest data is to be transmitted immediately. Therefore, when the sensor data processing unit 703 receives a command whose value in the “data acquisition timing” field is “sense”, the latest data specified in the “data type” field is immediately updated to the main control unit 800. Send to.

  The “event” designates transmission at a timing when a change occurs in the image data received from the sensor control unit 602. Therefore, when the sensor data processing unit 703 receives a command whose value in the “data acquisition timing” field is “event”, the image that receives the data specified in the “data type” field from the sensor control unit 602. The data is transmitted to the main control unit 800 at a timing when a change larger than a predetermined threshold occurs.

  “All” designates data transmission at a predetermined cycle. Therefore, when the sensor data processing unit 703 receives a command whose value in the “data acquisition timing” field is “all”, the data designated in the “data type” field is transferred to the main control unit 800 at a predetermined cycle. Send to. The predetermined period coincides with the period in which the optical sensor circuit 32 performs scanning.

  Next, the “data type” field is a field for designating the type of data acquired from the sensor data processing unit 703. In the “data type” field, for example, values of “001” (coordinates), “010” (partial image), and “100” (entire image) can be specified. Furthermore, by adding these values, “coordinates” and “partial image” / “whole image” can be specified simultaneously. For example, when “coordinate” and “partial image” are specified at the same time, “011” can be specified.

  When the sensor data processing unit 703 receives a command whose value of the “data type” field is “whole image”, the sensor data processing unit 703 transmits the image data itself stored in the image data buffer 704 to the main control unit 800. The image data itself stored in the image data buffer 704 is referred to as “whole image data”.

  In addition, when the sensor data processing unit 703 receives a command whose value of the “data type” field is “partial image”, the sensor data processing unit 703 selects a portion where a change larger than a predetermined threshold has occurred from the image data received from the sensor control unit 602. A region to be included is extracted, and image data of the extracted region is transmitted to the main control unit 800. Here, the image data is referred to as “partial image data”. When a plurality of partial image data are extracted, the sensor data processing unit 703 transmits each extracted partial image data to the main control unit 800.

  Further, when the sensor data processing unit 703 receives a command whose value of the “data type” field is “partial image”, the sensor data processing unit 703 detects representative coordinates in the partial image data and indicates the position of the representative coordinates in the partial image data. The coordinate data is transmitted to the main control unit 800. The representative coordinates include, for example, the coordinates of the center of the partial image data, the coordinates of the center of gravity of the partial image data, and the like.

  Next, when receiving a command whose value of the “data type” field is “coordinate”, the sensor data processing unit 703 transmits coordinate data indicating the position of the representative coordinate in the entire image data to the main control unit 800. When a plurality of partial image data are extracted, the sensor data processing unit 703 detects representative coordinates in the entire image data of the extracted partial image data, and the coordinate data indicating the representative coordinates is detected. Each is transmitted to the main control unit 800 (multi-point detection).

  Specific examples of the whole image data, the partial image data, and the coordinate data will be described later with reference to schematic diagrams.

  Next, the “scan method” field is a field for designating whether or not the backlight 307 is turned on at the time of executing the scan. In the “scan method” field, for example, values of “00” (reflection), “01” (transmission), and “10” (reflection / transmission) can be designated.

  “Reflection” specifies that scanning is performed with the backlight 307 turned on. Therefore, when the sensor data processing unit 703 receives a command whose “scan method” field value is “reflection”, the sensor data processing unit 703 performs sensor control so that the optical sensor driving circuit 305 and the backlight driving circuit 308 operate in synchronization. An instruction is given to the unit 602 and the backlight control unit 603.

  “Transmission” specifies that scanning is performed with the backlight 307 turned off. Therefore, when the sensor data processing unit 703 receives a command whose “scan method” field value is “transparent”, the sensor control unit 602 operates the optical sensor driving circuit 305 and does not operate the backlight driving circuit 308. Instructions to the backlight control unit 603. Note that “reflection / transmission” specifies that scanning is performed using both “reflection” and “transmission”.

  Next, the “scanned image gradation” field is a field for designating gradations of the partial image data and the entire image data. In the “scanned image gradation” field, for example, values of “00” (binary) and “01” (multivalue) can be designated.

  When the sensor data processing unit 703 receives a command whose “scan image gradation” field value is “binary”, the sensor data processing unit 703 transmits the partial image data and the entire image data to the main control unit 800 as monochrome data. .

  When the sensor data processing unit 703 receives a command whose “scanned image gradation” field value is “multivalued”, the sensor data processing unit 703 transmits the partial image data and the entire image data to the main control unit 800 as multitone data. To do.

  Next, the “scan resolution” field is a field for designating the resolution of the partial image data and the entire image data. In the “resolution” field, for example, values of “0” (high) and “1” (low) can be designated.

  Here, “high” designates a high resolution. Therefore, when the sensor data processing unit 703 receives a command whose “scan resolution” field value is “high”, the sensor data processing unit 703 transmits the partial image data and the entire image data to the main control unit 800 with high resolution. For example, it is desirable to designate “high” for image data (image data such as a fingerprint) to be subjected to image processing such as image recognition.

  “Low” designates a low resolution. Therefore, when the sensor data processing unit 703 receives a command whose “scan resolution” field value is “low”, the sensor data processing unit 703 transmits the partial image data and the entire image data to the main control unit 800 at a low resolution. For example, it is desirable to designate “low” for image data (such as touched finger or hand image data) that only needs to be recognized.

  Next, the “scan panel” field is a field for designating which display / light sensor unit 300 is to scan the object. In the “scan panel” field, for example, values “001” (first display / light sensor unit 300A) and “010” (second display / light sensor unit 300B) can be designated. Note that by adding these values, a plurality of display / light sensor units 300 can be specified simultaneously. For example, when “first display / light sensor unit 300A” and “second display / light sensor unit 300B” are specified at the same time, “011” can be specified.

  When the sensor data processing unit 703 receives a command whose “scan panel” field value is “first display / photosensor unit 300A”, the sensor data processing unit 703 and the photosensor drive circuit 305 of the first display / photosensor unit 300A and An instruction is given to the sensor control unit 602 and the backlight control unit 603 so as to control the backlight drive circuit 308.

  Next, the “display panel” field is a field for designating which display / light sensor unit 300 displays the display data. In the “scanned image gradation” field, for example, values of “001” (first display / light sensor unit 300A) and “010” (second display / light sensor unit 300B) can be designated. By adding these values, a plurality of display / light sensor units 300 can be specified at the same time. For example, when “first display / light sensor unit 300A” and “second display / light sensor unit 300B” are specified at the same time, “011” can be specified.

  Here, for example, when the display data processing unit 701 receives a command whose value of the “display panel” field is “first display / light sensor unit 300A”, the display data processing unit 701 displays the display data on the first display / light sensor unit 300A. Therefore, an instruction is given to the display control unit 601 and the backlight control unit 603 to control the liquid crystal panel driving circuit 304 and the backlight driving circuit 308 of the first display / light sensor unit 300A.

  Next, the “reserved” field is a field that is appropriately specified when it is necessary to further specify information other than information that can be specified in the above-described fields.

  Note that an application executed by the main control unit 800 does not need to use all the above-described fields when transmitting a command, and an invalid value (such as a NULL value) may be set for a field that is not used. .

  When the user wants to acquire coordinate data of a position touched with a finger or pen, a command specifying “coordinate” in the “data type” field is transmitted to the data processing unit 700. Therefore, it is desirable to specify “all” in the “data acquisition timing” field of the command to acquire coordinate data. Further, since it is only necessary to acquire coordinate data of the touched position, the scanning accuracy may not be high. Therefore, “low” may be specified as the value of the “resolution” field of the command.

  Further, when “coordinate” is specified in the “data type” field of the command, for example, when the user touches the sensor built-in liquid crystal panel 301 with a plurality of fingers or pens at the same time, the coordinate data of the touched position is used. Can be acquired (multi-point detection).

  When acquiring image data of an object such as a document, a command specifying “whole image” in the “data type” field is transmitted to the data processing unit 700. Since it is common to perform a scan in a stationary state, it is not necessary to periodically perform the scan. Therefore, in this case, it is desirable to designate “sense” or “event” in the “data acquisition timing” field. When scanning an object such as a document, it is desirable that the scanning accuracy is high so that the user can easily read the characters. Therefore, it is desirable to designate “high” in the “resolution” field.

(Whole image data / Partial image data / Coordinate data)
Next, the whole image data, the partial image data, and the coordinate data will be described with reference to FIG. The image data shown in FIG. 7A is image data obtained as a result of scanning the entire sensor built-in liquid crystal panel 301 when the object is not placed on the sensor built-in liquid crystal panel 301. The image data shown in FIG. 7B is image data obtained as a result of scanning the entire sensor-equipped liquid crystal panel 301 when the user touches the sensor-equipped liquid crystal panel 301 with a finger.

  When the user touches the sensor built-in liquid crystal panel 301 with a finger, the amount of light received by the photosensor circuit 32 in the vicinity of the touch changes, so that the voltage output from the photosensor circuit 32 changes, and as a result, In the image data generated by the sensor control unit 602, the brightness of the pixel value of the portion touched by the user changes.

  In the image data shown in FIG. 7B, the brightness of the pixel value of the portion corresponding to the user's finger is higher than that in the image data shown in FIG. In the image data shown in FIG. 7B, the smallest rectangular area (area PP) that includes all pixel values whose lightness changes more than a predetermined threshold is “partial image data”.

  The image data indicated by the area AP is “whole image data”.

  Also, the coordinate data in the whole image data (area AP) of the representative coordinates Z of the partial image data (area PP) is (Xa, Ya), and the coordinate data in the partial image data (area PP) is (Xp, Yp). It is.

(Configuration of sensor built-in liquid crystal panel)
Next, the configuration of the sensor built-in liquid crystal panel 301 and the configuration of the peripheral circuit 309 of the sensor built-in liquid crystal panel 301 will be described with reference to FIG. FIG. 8 is a block diagram showing the main part of the display / light sensor unit 300, particularly the configuration of the sensor built-in liquid crystal panel 301 and the configuration of the peripheral circuit 309.

  The sensor built-in liquid crystal panel 301 includes a pixel circuit 31 for setting light transmittance (brightness) and an optical sensor circuit 32 that outputs a voltage corresponding to the intensity of light received by the sensor. Note that the pixel circuit 31 is a generic term for the R pixel circuit 31r, the G pixel circuit 31g, and the B pixel circuit 31b corresponding to the red, green, and blue color filters, respectively.

  The pixel circuits 31 are arranged on the sensor built-in liquid crystal panel 301 in the column direction (vertical direction) and 3n in the row direction (horizontal direction). A set of the R pixel circuit 31r, the G pixel circuit 31g, and the B pixel circuit 31b is continuously arranged in the row direction (lateral direction). This set forms one pixel.

  In order to set the light transmittance of the pixel circuit 31, first, the high level voltage (TFT 33 is turned on) to the scanning signal line Gi connected to the gate terminal of the TFT (Thin Film Transistor) 33 included in the pixel circuit 31. Voltage). Thereafter, a predetermined voltage is applied to the data signal line SRj connected to the source terminal of the TFT 33 of the R pixel circuit 31r. Similarly, the light transmittance is also set for the G pixel circuit 31g and the B pixel circuit 31b. Then, by setting these light transmittances, an image is displayed on the sensor built-in liquid crystal panel 301.

  Next, the photosensor circuit 32 is arranged for each pixel. One pixel may be arranged in the vicinity of each of the R pixel circuit 31r, the G pixel circuit 31g, and the B pixel circuit 31b.

  In order for the optical sensor circuit 32 to output a voltage corresponding to the light intensity, first, the sensor readout line RWi connected to one electrode of the capacitor 35 and the anode terminal of the photodiode 36 are connected. A predetermined voltage is applied to the sensor reset line RSi. In this state, when light is incident on the photodiode 36, a current corresponding to the amount of incident light flows through the photodiode 36. Then, according to the current, the voltage at the connection point (hereinafter referred to as connection node V) between the other electrode of the capacitor 35 and the cathode terminal of the photodiode 36 decreases. When the power supply voltage VDD is applied to the voltage application line SDj connected to the drain terminal of the sensor preamplifier 37, the voltage at the connection node V is amplified and output from the source terminal of the sensor preamplifier 37 to the sensing data output signal line SPj. . Based on the output voltage, the amount of light received by the optical sensor circuit 32 can be calculated.

  Next, the liquid crystal panel drive circuit 304, the optical sensor drive circuit 305, and the sensor output amplifier 44, which are peripheral circuits of the sensor built-in liquid crystal panel 301, will be described.

  The liquid crystal panel drive circuit 304 is a circuit for driving the pixel circuit 31, and includes a scanning signal line drive circuit 3041 and a data signal line drive circuit 3042.

  The scanning signal line driving circuit 3041 sequentially selects one scanning signal line from the scanning signal lines G1 to Gm for each line time based on the timing control signal TC1 received from the display control unit 601, and A high level voltage is applied to the selected scanning signal line, and a low level voltage is applied to the other scanning signal lines.

  Based on the display data D (DR, DG, and DB) received from the display controller 601, the data signal line driver circuit 3042 generates a predetermined voltage corresponding to the display data for one row for each line time. The data signal lines SR1 to SRn, SG1 to SGn, and SB1 to SBn are applied (line sequential method). Note that the data signal line driver circuit 3042 may be driven by a dot sequential method.

  The optical sensor driving circuit 305 is a circuit for driving the optical sensor circuit 32. Based on the timing control signal TC2 received from the sensor control unit 602, the optical sensor driving circuit 305 selects a predetermined sensor readout signal line from the sensor readout signal lines RW1 to RWm for each line time. A read voltage is applied, and a voltage other than a predetermined read voltage is applied to the other sensor read signal lines. Similarly, based on the timing control signal TC2, a predetermined reset voltage is applied to the sensor reset signal line selected from the sensor reset signal lines RS1 to RSm for each line time, and the others. A voltage other than a predetermined reset voltage is applied to the sensor reset signal line.

  The sensing data output signal lines SP1 to SPn are grouped into p groups (p is an integer of 1 to n), and the sensing data output signal lines belonging to each group are connected via a switch 47 that is sequentially turned on in time division. And connected to the sensor output amplifier 44. The sensor output amplifier 44 amplifies the voltage from the group of sensing data output signal lines connected by the switch 47 and outputs the amplified voltage to the signal conversion circuit 306 as sensor output signals SS (SS1 to SSp).

(Data display / sensor device housing)
Next, the housing of the data display / sensor device 100 according to the present invention will be described with reference to FIGS. 9 (a) and 9 (b). FIG. 9A and FIG. 9B are schematic views showing the housing of the data display / sensor device 100. FIG. As shown in the figure, the data display / sensor device 100 is a so-called clamshell type housing that can be folded at the hinge portion H so that the first display / light sensor portion 300A and the second display / light sensor portion 300B face each other. Have a body.
FIG. 9A is a diagram illustrating a state in which the housing of the data display / sensor device 100 is opened (that is, not folded), and FIG. 9B is a diagram illustrating the housing of the data display / sensor device 100. It is a figure which shows a mode that it closed (that is, it folded).

  As shown in FIG. 9A, when the housing of the data display / sensor device 100 is opened, the sensor built-in liquid crystal panel 301 of the first display / light sensor unit 300A and the sensor of the second display / light sensor unit 300B are built. At least one of the liquid crystal panel 301 can be seen by the user, and data display and user input are possible.

  Further, as shown in FIG. 9B, when the housing of the data display / sensor device 100 is closed, the sensor built-in liquid crystal panel 301 of the first display / light sensor unit 300A and the second display / light sensor unit 300B Since the sensor-equipped liquid crystal panel 301 faces the sensor-incorporated liquid crystal panel 301, it is possible to scan one or both sides of the object by sandwiching the object such as the document P1 between both the sensor-incorporated liquid crystal panels 301. .

  When the housing of the data display / sensor device 100 is closed, as shown in FIG. 9B, either the first display / light sensor unit 300A or the second display / light sensor unit 300B is on the upper side. Yes, the other is on the lower side. Therefore, hereinafter, for convenience of explanation, the first display / light sensor unit 300A is also referred to as an “upper screen”, and the second display / light sensor unit 300B is also referred to as a “lower screen”. The display / light sensor unit 300 that executes scanning is also referred to as a “scanning execution surface”.

  Further, as shown in FIG. 9A, the description will be made assuming that the user usually places an object such as a document P1 on the sensor built-in liquid crystal panel 301 of the second display / light sensor unit 300B which is the lower screen.

  Hereinafter, the sensor built-in liquid crystal panel 301 of the first display / light sensor unit 300A is referred to as a sensor built-in liquid crystal panel 301A, and the sensor built-in liquid crystal panel 301 of the second display / light sensor unit 300B is referred to as a sensor built-in liquid crystal panel. This is expressed as 301B. When the sensor built-in liquid crystal panel 301A and the sensor built-in liquid crystal panel 301B are not distinguished, they are referred to as a sensor built-in liquid crystal panel 301.

  Further, hereinafter, when simply “display (or display)” is described, it is displayed (or displayed) on the sensor built-in liquid crystal panel 301.

(Key points of the invention)
A main feature of the present invention is that a predetermined image (for example, a predetermined mark or a pattern image) is displayed on one of the sensor built-in liquid crystal panel 301A and the sensor built-in liquid crystal panel 301B of the data display / sensor device 100. In the state, when the predetermined image can be read by the sensor built-in liquid crystal panel 301 on the side on which the predetermined image is not displayed, it is determined that the casing is closed, and the predetermined image is displayed in advance. It is to execute the associated processing. Hereinafter, the predetermined image is referred to as a “detection image”.

  For example, a case will be described in which processing for executing scanning of an object is associated in advance with a detection image. In this case, for example, when the detection image can be read by the sensor built-in liquid crystal panel 301B in a state where the detection image is displayed on the sensor built-in liquid crystal panel 301A, the sensor built-in liquid crystal panel 301A and the sensor built-in The scanning of the object sandwiched between the liquid crystal panel 301B is executed by at least one of the sensor built-in liquid crystal panel 301A and the sensor built-in liquid crystal panel 301B.

  When the housing of the data display / sensor device 100 is closed, the sensor built-in liquid crystal panel 301A and the sensor built-in liquid crystal panel 301B are in contact with each other. When scanning is performed, as a result of the scanning, it is possible to acquire image data including an image displayed on the sensor built-in liquid crystal panel 301B. In other words, when image data including an image displayed on the sensor built-in liquid crystal panel 301B can be acquired as a result of scanning with the sensor built-in liquid crystal panel 301A, the housing of the data display / sensor device 100 is displayed. Can be determined to be closed.

  Therefore, when the detection image can be read, it may be determined that the housing of the data display / sensor device 100 is closed.

  Therefore, it is possible to detect that the casing is closed without depending on a mechanical switch such as the hinge switch 904, and to scan the object.

  Further, if the user can instruct whether or not to display the detection image on the sensor built-in liquid crystal panel 301, the user gives an instruction to display the detection image on the sensor built-in liquid crystal panel 301. The object can be scanned only when the casing is closed. In other words, if the user closes the housing without giving an instruction to display the detection image on the sensor built-in liquid crystal panel 301, the data display / sensor device 100 is prevented from executing scanning of the object. Is possible. In other words, since the scan of the object is not executed when the user simply closes the casing, the situation where the scan is erroneously executed by the user closing the casing is prevented. Can do.

  Next, a case where processing for turning off the power of the data display / sensor device 100 is associated with the detection image in advance will be described. In this case, when the detection image can be read by the sensor built-in liquid crystal panel 301B in a state where the detection image is displayed on the sensor built-in liquid crystal panel 301A, for example, the power source of the data display / sensor device 100 Turn off.

  Therefore, for example, (a) when the user closes the casing with the object placed on the sensor-embedded liquid crystal panel 301A so as to overlap the detection image, the entire detection image is read. Therefore, the data display / sensor device 100 is not turned off. (B) The user does not place the object on the sensor-equipped liquid crystal panel 301A displaying the detection image. When the casing is closed, the entire detection image can be read, so that the data display / sensor device 100 can be turned off.

  In addition, compared to a method of performing a scan when detecting that the casing is closed by a mechanical switch such as a hinge switch 904, simply closing the casing causes the object to be Since scanning is not executed, unnecessary scanning can be prevented.

[Embodiment 1]
First, the form which performs the scan of a target object by reading the image for a detection is demonstrated. One embodiment of the present invention is described below with reference to FIG. 1 and FIGS. 10 to 18.

(Scanning the object by reading the detection image)
A typical example of processing for executing scanning of an object by reading a detection image will be described with reference to FIGS. 10 (a) to 10 (e). Here, an outline of the processing will be described, and details of each processing will be described later.

  First, FIG. 10A is a diagram illustrating a state in which the data display / sensor device 100 displays a message that prompts the user to prepare for scanning. Here, a region (hereinafter referred to as a placement region) R1 on which an object is placed for scanning is displayed on the sensor built-in liquid crystal panel 301B on the lower screen and a message to the user is displayed on the sensor built-in liquid crystal panel 301A on the upper screen. M1 is displayed.

  Next, FIG. 10B is a diagram showing a state in which a document P2 (for example, a business card) as an object is placed on the sensor built-in liquid crystal panel 301B on the lower screen. At this time, when the sensor built-in liquid crystal panel 301B detects that the document P2 is placed in the placement area R1, the button BT1 operable by the user is displayed on the sensor built-in liquid crystal panel 301B, and the sensor built-in liquid crystal panel 301A is displayed. A new message M2 is displayed.

  Here, the button BT1 is a button for receiving an instruction to display a detection image J1 described later from the user. In FIG. 5B, a character string “Ready?” Is displayed on the button BT1, and the user touches the button BT1 when scanning is ready by the character string and the message M2. Urging. It should be noted that the display position of the button BT1 is desirably a position that is in the vicinity of the placement area R1 and does not overlap the placement area R1. The button BT1 may be displayed on the sensor built-in liquid crystal panel 301A on the upper screen.

  Note that when the user touches the button BT1, the data display / sensor device 100 causes the detection-use image J1 associated with the process of executing the scan of the object in the sensor-containing liquid crystal panel 301A and the sensor-containing liquid crystal panel 301B. Is displayed on the sensor built-in liquid crystal panel 301B.

  Next, FIG.10 (c) is a figure which shows the mode after a user touches button BT1 in the state shown in FIG.10 (b). At this time, instead of the button BT1, the data display / sensor device 100 displays the detection image J1 on the sensor built-in liquid crystal panel 301B and also displays a new message M3 on the sensor built-in liquid crystal panel 301A. In FIG. 3C, a character string “Go!” Is displayed in the detection image J1, and the casing of the data display / sensor device 100 is closed to the user by the character string and the message M3. I encourage you to.

  The shape and color of the detection image J1 are preferably different from the shape and color of the button BT1, respectively, and the detection image J1 is preferably displayed at the position where the button BT1 was displayed. . This is to make it easier for the user to visually recognize that the detection image J1 is displayed instead of the button BT1.

  Next, FIG. 10D is a diagram illustrating a state in which the housing of the data display / sensor device 100 is closed in the state illustrated in FIG. At this time, the sensor built-in liquid crystal panel 301A performs scanning in order to read the detection image J1. When it is detected that the image data obtained as a result of the scan includes an area corresponding to the detection image J1, the data display / sensor device 100 determines that the casing is closed. It is desirable that the shape and color of the detection image J1 be easy to detect that the image data obtained as a result of the scan includes a region corresponding to the detection image J1.

  If it is determined that the housing of the data display / sensor device 100 is closed, the surface of the document P2 is scanned by the sensor built-in liquid crystal panel 301A on the upper screen as processing associated with the detection image J1. At the same time, the back side of the document P2 is scanned by the sensor built-in liquid crystal panel 301B on the lower screen. When the scan is completed, the user is notified that the scan is complete (for example, a predetermined sound is output or a predetermined lamp is lit).

  Next, FIG. 10E is a diagram illustrating a state in which the housing of the data display / sensor device 100 is opened after the scan is executed in the state illustrated in FIG. The document P2 is removed from the sensor built-in liquid crystal panel 301 on the lower screen as shown in the figure. Then, the sensor built-in liquid crystal panel 301A on the upper screen is a state in which the scan data SCD1, which is a portion corresponding to the surface of the document P2, among the image data obtained as a result of scanning with the sensor built-in liquid crystal panel 301A is rotated 180 degrees Is displayed. Further, the sensor built-in liquid crystal panel 301B on the lower screen displays scan data SCD2, which is a portion corresponding to the back side of the document P2, among the image data obtained as a result of scanning with the sensor built-in liquid crystal panel 301B.

  It should be noted that a character recognition function is provided in order to change the display orientation and display position of the scan data SCD1 and SCD2 by the user operation, store the scan data SCD1 and SCD2 in the storage unit 901, and recognize the characters included in the scan data SCD1 and SCD2. It may be activated. The user operation is performed by touching a menu (not shown) or directly touching the scan data SCD1 and SCD2.

  In FIG. 10E, the placement region R1 and the detection image J1 are not displayed, but may remain displayed. Thus, the new document can be scanned simply by placing the new document on the placement area R1 and closing the housing of the data display / sensor device 100. That is, the scanning of a plurality of documents is continuously performed only by repeating the operation of placing the document that is the object of scanning on the placement region R1 and the operation of opening and closing the housing of the data display / sensor device 100. It can be carried out.

  In the above description, the example in which the detection image J1 is displayed by the user touching the button BT1 has been described. However, when the user touches a predetermined area of the sensor built-in liquid crystal panel 301 without displaying the button BT1. The detection image J1 may be displayed.

  In the following description, a portion corresponding to the object in the image data obtained as a result of scanning the object is referred to as “scan data”.

(Data display / more detailed configuration of sensor device)
Next, a more detailed configuration of the data display / sensor device 100 will be described with reference to FIG. Here, in order to make the description easy to understand, the description of the operations of the data processing unit 700 and the circuit control unit 600 located between the main control unit 800 and the display / light sensor unit 300 is omitted. However, to be precise, when displaying data and scanning an object, each unit of the main control unit 800 transmits a command to the data processing unit 700, and the data processing unit 700 sets the circuit control unit 600 based on the command. Then, the circuit control unit 600 transmits a signal to the display / light sensor unit 300. Further, the main control unit 800 acquires the entire image data, the partial image data, and the coordinate data from the data processing unit 700 as a response to the command transmitted to the data processing unit 700.

  FIG. 1 is a block diagram showing a more detailed configuration of the data display / sensor device 100. As illustrated, the storage unit 901 includes a menu storage unit 21, a scan attribute information storage unit 22, an image data storage unit 23, and a detection image storage unit 24.

  The menu storage unit 21 stores information related to a menu displayed on the sensor built-in liquid crystal panel 301. The menu is a list composed of a plurality of menu items that can be selected by the user. The menu item is associated with a process to be executed when a user is selected.

  The menu storage unit 21 stores a menu corresponding to the operation state of the data display / sensor device 100. For example, immediately after the data display / sensor device 100 is started, as a menu displayed on the sensor built-in liquid crystal panel 301, the menu item “image scan” associated with the scan process is stored in the storage unit. “Image display”, which is a menu item associated with a process for displaying image data on the sensor-equipped liquid crystal panel 301, and “Address book”, a menu item associated with a process for starting an address book application The menu that contains etc. is memorized.

  In addition, for example, in a state where image data is displayed on the sensor built-in liquid crystal panel 301, menus displayed on the sensor built-in liquid crystal panel 301 include “enlarge / reduce”, “full screen display”, “rotate / invert” ”,“ Upper / Lower Display Swap ”,“ Edit ”,“ Character Recognition ”,“ Save to External Storage ”,“ Various Settings ”,“ Exit ”, and other menu items are stored.

  The menu item may be further associated with a sub menu item. For example, the “edit” menu item may be associated with “add comment”, “change display range”, “highlight pen”, “black / white reversal”, and the like as submenu items.

  The scan attribute information storage unit 22 stores various types of information used by the scan processing unit 12 to scan an object placed on the sensor built-in liquid crystal panel 301. The scan attribute information storage unit 22 includes a display message storage unit 221, a scan pattern storage unit 222, and a scan mode storage unit 223.

  The display message storage unit 221 includes (1) a predetermined message to be presented to the user, and (2) information indicating which of the sensor built-in liquid crystal panel 301A and the sensor built-in liquid crystal panel 301B is to be displayed (hereinafter, message display). And (3) a set of information (coordinate data etc.) (hereinafter, message display area information) indicating an area for displaying the message on the sensor built-in liquid crystal panel 301 is stored in a readable state. Hereinafter, the group stored in the display message storage unit 221 is referred to as message information.

  As a predetermined message to be presented to the user, for example, “Place paper at the scan position”, which is a message for prompting preparation of the object, and touching a button on the screen when the object is ready. “Please touch the button on the screen when ready” message to prompt, “Please close the lid” message to prompt the user to close the case, message to prompt confirmation of the scan result Character data such as “Please check the scan result” can be mentioned. The message is not limited to character data but may be image data.

  Next, the scan pattern storage unit 222 includes (1) information indicating a placement area (hereinafter, placement area information), and (2) any of the first display / light sensor unit 300A and the second display / light sensor unit 300B. (3) whether the scan data is displayed on the first display / light sensor unit 300A or the second display / light sensor unit 300B. A set of information (hereinafter referred to as scan data display surface information) is stored in a readable state. Hereinafter, the set stored in the scan pattern storage unit 222 is also referred to as scan pattern information.

  The information (1) to (3) included in the scan pattern information may be set in advance at the time of factory shipment or can be set by the user via the scan attribute information setting unit 138. It may be.

  Also, any one of the scan pattern information stored in the scan pattern storage unit 222 is referred to by the scan processing unit 12. Which scan pattern information is to be referred to may be set in advance at the time of factory shipment or the like, or may be set by the user via the scan attribute information setting unit 138.

  The scan pattern storage unit 222 can have a data structure as shown in Table 1 below, for example. Table 1 is a table showing an example of the data structure of the scan pattern storage unit 222.

  As shown in Table 1, the scan pattern storage unit 222 stores a set of “placement area”, “scan execution surface”, and “scan data display surface”.

  The “placement area” is a column that stores placement area information. As described above, here, the object is placed on the sensor built-in liquid crystal panel 301 on the lower screen. Therefore, in the example of Table 1, in the sensor built-in liquid crystal panel 301 on the lower screen, a rectangular area in which the upper left coordinate is (x3, y3) and the lower right coordinate is (x4, y4) is placed. It is stored as an area.

  The “scan execution surface” is a column for storing scan execution surface information. Therefore, as shown in the first record in Table 1, when the “scan execution surface” is “upper screen”, this means that the sensor built-in liquid crystal panel 301A executes the scan. Further, as in the second record in Table 1, when the “scan execution surface” is “lower screen”, this means that the sensor built-in liquid crystal panel 301B executes the scan. As shown in the third record of Table 1, when the “scan execution surface” is “upper screen” and “lower screen”, both the sensor built-in liquid crystal panel 301A and the sensor built-in liquid crystal panel 301B perform scanning. It means to execute.

  The “scan data display surface” is a column for storing scan data display surface information. Therefore, as shown in the first record of Table 1, when the “display surface” is “upper screen”, this means that the scan data is displayed on the first display / light sensor unit 300A. Even when a plurality of scan data is obtained as a result of placing a plurality of objects on the placement area, all the scan data is displayed on the upper screen.

  If the scan data is not displayed, the “scan data display surface” column value may be set to a NULL value or the like.

  Here, the scan pattern information stored in Table 1 will be described for each record. The first record in Table 1 is: (1) The placement area is a rectangular area whose upper left coordinates are (x3, y3) and lower right coordinates are (x4, y4) on the lower screen. , (2) scan pattern information is stored in which scanning is executed on the upper screen, and (3) scan data is displayed on the upper screen.

  Next, the second record in Table 1 shows that (1) the loading area is a rectangle whose upper left coordinates are (x3, y3) and lower right coordinates are (x4, y4) on the lower screen. This area stores scan pattern information that (2) scan is executed on the lower screen, and (3) scan data is displayed on the upper screen.

  Next, the third record in Table 1 shows that (1) the loading area is a rectangle whose upper left coordinates on the lower screen are (x3, y3) and whose lower right coordinates are (x4, y4). This area stores scan pattern information in which (2) scanning is executed on the upper screen and lower screen, and (3) any scan data is displayed on the upper screen.

  Finally, the fourth record in Table 1 shows that (1) the loading area is a rectangle whose upper left coordinates are (x3, y3) and lower right coordinates are (x4, y4) on the lower screen. (2) Scan on the upper screen and lower screen, (3) Scan data obtained as a result of scanning on the upper screen is displayed on the upper screen, and scan obtained as a result of scanning on the lower screen Scan pattern information for displaying data on the lower screen is stored.

  Note that the shape of the area stored in the “mounting area” is not limited to a rectangle, and may be, for example, a circle. Further, the information indicating the area is not limited to the coordinates.

  Next, the scan mode storage unit 223 stores information (hereinafter referred to as scan mode information) indicating whether or not to continuously perform scanning. When it is stored that scanning is continuously executed, scanning is continuously executed with a predetermined user operation (for example, data display / opening / closing operation of the housing of the sensor device 100) as a trigger. Shall.

  Next, the image data storage unit 23 stores image data such as scan data in a readable state. A comment input by the user may be stored in association with the image data.

  Next, the detection image storage unit 24 stores the detection image in a readable state. Further, the detection image storage unit 24 stores the detection image and processing designation information indicating processing to be executed by the own apparatus when the detection image is read in association with each other. Examples of the process to be executed include (1) scanning of an object, (2) turning off the power of the data display / sensor device 100, and the like. In the above (1), double-sided scanning / single-sided scanning of an object, scanning of only characters other than a specific color of the object, and the like may be specified as parameters.

  The association may be stored using, for example, the table structure shown in Table 2 below. Table 2 is an example of a table structure for storing the detection image and the processing designation information in association with each other. In Table 2, the file name for identifying the detection image is associated with the processing designation information. For example, the detection designation identified by the file name “mark001.jpg” is associated with processing designation information “scanning object” that is information indicating processing for executing scanning by the sensor built-in liquid crystal panel 301. ing. In addition, the detection image identified by the file name “mark002.jpg” is associated with processing designation information “power off”, which is information indicating processing for turning off the power of the data display / sensor device 100. Yes.

  The association between the detection image and the processing designation information may be set in advance at the time of factory shipment or may be set by the user.

  Next, the main control unit 800 will be described. As illustrated, the main control unit 800 includes a menu display control unit 11, a scan processing unit 12, an image display processing unit 18, and a power management unit 19 (processing execution unit). When image display processing is not performed, the image display processing unit 18 may not be included.

  The menu display control unit 11 reads menus and buttons corresponding to the operation state of the data display / sensor device 100 from the menu storage unit 21 and displays the read menus and buttons on the sensor built-in liquid crystal panel 301.

  When the user operates (touches) the sensor built-in liquid crystal panel 301 with a finger or a pen, the optical sensor circuit 32 detects at least one of a shadow image and a reflected image of the finger and the pen. The menu display control unit 11 can acquire coordinate data indicating the touched position.

  Therefore, when the position indicated by the obtained coordinate data is within the area where the menu or button is displayed on the sensor built-in liquid crystal panel 301, the menu display control unit 11 is the one in which the user operates the menu or button. Can be determined.

  Therefore, when the acquired coordinate data is the coordinate data corresponding to the position where the menu item is displayed, the menu display control unit 11 executes a process associated with the menu item. For example, when the coordinate data corresponding to the position where the menu item “image scan” is displayed is received, the scan processing unit 12 is activated to perform the scan process. When the coordinate data corresponding to the position where the menu item “image display” is displayed is received, the image display processing unit 18 is activated to execute the image display processing.

  Next, the scan processing unit 12 scans one or both sides of the object placed on the sensor built-in liquid crystal panel 301. For this purpose, the scan processing unit 12 includes a pre-scan processing unit 121, a detection image scanning unit 126 (processing execution unit), an object scan execution unit 131 (processing execution unit), and a scan attribute information setting unit 138. .

  The pre-scan processing unit 121 performs processing for prompting the user to prepare for scanning and determining whether or not the object is placed in the placement area. In addition, processing for displaying a detection image on the sensor built-in liquid crystal panel 301 is performed. For this purpose, the pre-scan processing unit 121 includes a placement area information presentation unit 122, an object presence / absence determination unit 123, and a detection image display processing unit 124 (detection image display instruction unit).

  The placement area information presentation unit 122 presents the placement area to the user and presents a message for prompting the preparation of the object to the user. For this purpose, the placement area information presentation unit 122 reads the placement area information included in the scan pattern information to be referenced from the scan pattern storage unit 222, and the above-described placement in the sensor built-in liquid crystal panel 301 on the lower screen. For example, a predetermined pattern image is displayed in the area indicated by the area information.

  Note that it is desirable to blink the predetermined pattern image. By blinking the display, it is possible to notify the user that the data display / sensor device 100 is ready to execute the scanning process and to notify the position of the placement area.

  Further, the placement area information presentation unit 122 refers to the display message storage unit 221 and acquires message information including a message for prompting the user to prepare the target object. Then, the message is displayed on the sensor built-in liquid crystal panel 301.

  Next, the object presence / absence determination unit 123 determines whether or not the object is placed in the placement area. For this purpose, the object presence / absence determination unit 123 acquires coordinate data indicating at least a position where the object is placed from the data processing unit 700. Furthermore, partial image data (including coordinate data in the partial image) indicating the region of the object may be acquired.

  When the object is placed on the sensor built-in liquid crystal panel 301, the optical sensor circuit 32 detects at least one of a shadow image and a reflected image of the object, and as a result, a sensor data processing unit of the data processing unit 700 is detected. A change occurs in the image data received by the sensor control unit 602 in step 703. As a result, the object presence / absence determining unit 123 receives coordinate data indicating the position where the object is placed and partial image data (including coordinate data in the partial image) indicating the area of the object from the data processing unit 700. Can be acquired.

  Therefore, the object presence / absence determination unit 123 checks whether the coordinate position indicated by the acquired coordinate data is included in the placement area. Thereby, the object presence / absence determination unit 123 determines whether or not the object is placed in the placement area. Further, when partial image data (including coordinate data in the partial image) is acquired, whether or not the area where the object is located on the liquid crystal panel with a built-in sensor 301 is within the placement area (or the placement area). Whether or not the object is placed in the placement area may be determined by examining whether or not the placement area overlaps the placement area.

  When it is determined that the object is placed in the placement area, the object presence / absence determination unit 123 refers to the display message storage unit 221 to prompt the user to instruct display of the detection image. Acquire message information including a message (for example, “Touch the screen when ready” or “Touch a button on the screen when ready”). Then, the message is displayed on the sensor built-in liquid crystal panel 301.

  Next, the detection image display processing unit 124 reads the detection image from the detection image storage unit 24 and displays the read detection image on the sensor built-in liquid crystal panel 301. Note that the detection image is desirably displayed when the casing of the data display / sensor device 100 is not closed (when the data display / sensor device 100 is not folded) so that the user can easily recognize the image. Examples of the timing for displaying the detection image include when the object presence / absence determining unit 123 determines that the object is placed in the placement area.

  The detection image display processing unit 124 receives a user operation, reads out the detection image from the detection image storage unit 24 using the received user operation as a trigger, and reads the read detection image into the sensor built-in liquid crystal panel 301. You may make it display on. The user operation is, for example, an operation in which the user touches a predetermined button or the like displayed on the sensor built-in liquid crystal panel 301, or a user touches a predetermined area in which the button or the like of the sensor built-in liquid crystal panel 301 is not displayed. It is assumed that the operations are equal.

  In addition, a predetermined detection image may be read from the detection image storage unit 24 or may be selectable by the user from those stored in the detection image storage unit 24. .

  As an example of displaying a predetermined detection image using a user operation as a trigger, as described in FIG. 10B, for receiving an instruction to display a detection image on the sensor built-in liquid crystal panel 301. A case where the button BT1 is displayed and the user touches the button BT1 to display the detection image J1 associated with the button BT1 can be considered. In this case, the detection image display processing unit 124 displays the button BT1 on the sensor built-in liquid crystal panel 301 when the target object presence / absence determination unit 123 determines that the target object is placed in the placement area. When detecting that the user touches the button BT1, the detection image J1 is displayed instead of the button BT1.

  Note that the display position and display size of the detection image displayed on the sensor built-in liquid crystal panel 301 by the detection image display processing unit 124 may be changed by a user operation. In this case, the user operation is performed, for example, by directly touching the detection image.

  Next, the detection image scanning unit 126 first performs scanning with at least one of the sensor built-in liquid crystal panel 301A and the sensor built-in liquid crystal panel 301B. Specifically, after the detection image is displayed on at least one of the sensor built-in liquid crystal panels 301, the detection image scanning unit 126 applies the detection image scanning unit 126 to the sensor built-in liquid crystal panel 301 on the side where the detection images are not displayed. Then, it is determined whether or not the image data obtained as a result of executing the scan includes a region corresponding to the detection image.

  Here, when the image data acquired as a result of scanning by the detection image scanning unit 126 is the entire image data, the region extracted from the entire image data by general image processing is matched with the detection image. Thus, it may be determined whether or not the entire image data includes an area corresponding to the detection image. When the image data acquired as a result of scanning by the detection image scanning unit 126 is one or a plurality of partial image data, the partial image data is matched by matching the partial image data with the detection image. Whether or not is an area corresponding to the detection image may be determined.

  Note that the above-mentioned matching is the shape or pattern of the image to be matched (the area corresponding to the detection image displayed on the sensor built-in liquid crystal panel 301 and the detection image in the image data acquired as a result of scanning). Based on the feature amount such as the coordinate position, a generally known matching method may be used.

  When the detection image scanning unit 126 determines that the image data obtained as a result of the scan includes a region corresponding to the detection image, the detection image storage unit 24 stores the detection. The process indicated by the process designation information associated with the image for use is executed. Specifically, if the processing designation information associated with the detection image is to execute scanning of an object, an instruction to drive the object scan execution unit 131 is given. Further, if the processing associated with the detection image is to turn off the power of the data display / sensor device 100, the power management unit 19 is instructed to turn off the power.

  When the housing of the data display / sensor device 100 is closed, the sensor built-in liquid crystal panel 301A and the sensor built-in liquid crystal panel 301B are in contact with each other. When scanning is performed, as a result of the scanning, it is possible to acquire image data including an image displayed on the sensor built-in liquid crystal panel 301B. In other words, when image data including an image displayed on the sensor built-in liquid crystal panel 301B can be acquired as a result of scanning with the sensor built-in liquid crystal panel 301A, the housing of the data display / sensor device 100 is displayed. May be determined to be closed. Therefore, when the detection image can be read by at least one of the sensor built-in liquid crystal panels 301, the detection image scanning unit 126 closes the casing of the data display / sensor device 100 (folded). Is determined).

  In addition, when it is determined that the image data obtained as a result of the scan includes an area corresponding to the detection image, the user may be notified of this (for example, a predetermined sound is output or a predetermined sound is output). The lamp lights up).

  If the image data obtained as a result of the scan does not include an area corresponding to the detection image, or includes only an area not corresponding to the detection image, the detection image scanning unit 126 is previously stored. A predetermined operation may be performed, or nothing may be performed until a user operation is performed.

  Next, the object scan execution unit 131 acquires the entire image data as a result of causing the sensor built-in liquid crystal panel 301 to execute the scan, extracts the scan data from the acquired entire image data, and extracts the scan data. It is stored in the storage unit. For this purpose, the object scan execution unit 131 includes a scan execution instruction unit 132, a scan data generation unit 133, a scan data display instruction unit 135, and a scan data storage unit 137.

  The scan execution instructing unit 132 causes the sensor built-in liquid crystal panel 301 to execute a scan in order to execute a scan of the object. At this time, the scan execution instructing unit 132 reads the reference scan pattern information from the scan pattern storage unit 222, and incorporates a sensor that executes the scan based on the scan execution surface information included in the read scan pattern information. The liquid crystal panel 301 is determined. Then, the scan execution instruction unit 132 causes the sensor built-in liquid crystal panel 301 to execute a scan.

  Next, when the scan data generation unit 133 acquires the entire image data as a result of the scan, the scan data generation unit 133 extracts a region of the target object as scan data from the acquired entire image data. When a plurality of objects are scanned, an area corresponding to each object is extracted as scan data.

  Here, as a method for extracting the region of the target object, for example, the target image region and the non-target object region are discriminated by general image processing on the acquired entire image data. There is a method of extracting an object area as scan data. In the acquired whole image data, the area of the object and the area of the part that is not the object are different in the brightness of the pixel value, and can be distinguished.

  Further, when the object presence / absence determination unit 123 acquires coordinate data indicating the position where the object is placed and partial image data (including coordinate data in the partial image) indicating the area of the object, scanning is performed. The data generation unit 133 may calculate a region of the object in the acquired entire image data and extract the region as scan data from the acquired entire image data. Specifically, based on the coordinate data acquired by the object presence / absence determination unit 123 and the coordinate data in the partial image, the position of the partial image data acquired by the object presence / absence determination unit 123 in the entire image data is determined. A region corresponding to the partial image data at the determined position is extracted as scan data from the entire image data determined and acquired as a result of scanning.

  Note that the area where the object is placed on the lower screen and the area where the object is scanned on the upper screen are in opposite positions (in a mirror image relationship). The area of the object in the obtained image data can be calculated based on the area of the object in the image data obtained as a result of scanning on the lower screen. Therefore, the scan data generation unit 133 scans on the upper screen based on the coordinate data indicating the position where the object is placed and the partial image data (including the coordinate data in the partial image) indicating the area of the object. The region of the object can be extracted as scan data also from the entire image data obtained as a result.

  Further, the scan data generation unit 133 (1) turns off the backlight 307 on the upper screen, turns on the backlight 307 on the lower screen, performs scanning on the upper screen, and (2) backs the lower screen. By turning off the light 307 and turning on the backlight 307 on the upper screen while executing scanning on the lower screen, coordinate data indicating the position where the object is placed and partial image data indicating the area of the object (Including coordinate data in the partial image) may be acquired.

  If the area of the object cannot be recognized, scan data is extracted from the acquired whole image data based on the scan data area information included in the scan pattern information read from the scan pattern storage unit 222. Also good.

  When the casing is closed, the optical sensor circuit 32 on the upper screen detects at least one of a shadow image and a reflected image of the object, and as a result, the sensor data processing unit 703 of the data processing unit 700 performs the sensor control unit. A change occurs in the image data received from 602. Therefore, the coordinate data indicating the position of the object scanned on the upper screen and the partial image data (including the coordinate data in the partial image) indicating the area of the object may be acquired from the data processing unit 700. Is possible. In this case, the acquired partial image data is scan data as a result of scanning on the upper screen. In this case, the scan data generation unit 133 calculates the target object from the entire image data obtained as a result of scanning on the lower screen based on the coordinate data and the partial image data (including the coordinate data in the partial image). An area can be extracted as scan data.

  Next, the scan data generation unit 133 primarily stores the extracted scan data in the scan data holding unit 136.

  Note that the object scan execution unit 131 temporarily stores the scan data in the scan data holding unit 136 after the scan data generation unit 133 temporarily stores the scan data, and then notifies the audio output unit 908 that the scan has ended. Output audio data.

  Next, the scan data display instruction unit 135 reads the scan data temporarily stored in the scan data holding unit 136 and displays it on the sensor built-in liquid crystal panel 301.

  In displaying the scan data, the scan data display instruction unit 135 reads the scan pattern information to be referenced from the scan pattern storage unit 222, and based on the scan data display surface information included in the read scan pattern information. Then, the sensor built-in liquid crystal panel 301 for displaying the scan data is determined, and the scan data is displayed on the determined sensor built-in liquid crystal panel 301.

  For example, the scan data display instruction unit 135 refers to the display message storage unit 221 with the detection that the hinge unit switch 904 has opened the housing as a trigger, and notifies the user that the scan has been completed. A message to the effect (for example, “scanning is completed”) is read, and the read message is displayed on the sensor built-in liquid crystal panel 301.

  In addition, if the scan data obtained as a result of scanning the object placed on the lower screen on the upper screen is displayed as it is, the image is displayed in an upside down state. Therefore, when displaying the scan data obtained as a result of executing the scan on the upper screen, it is desirable to display it in a state rotated by 180 degrees. In this case, the scan execution instruction unit 132 may perform image processing for rotating the scan data by 180 degrees, and display the scan data after the image processing on the sensor built-in liquid crystal panel 301.

  Also, the user can change the scan data display surface and the scan data display area for displaying the scan data by operating the scan data itself displayed in the scan data display area, or by operating a predetermined menu or button. The display orientation may be changeable.

  Next, the scan data storage unit 137 is set to continuously execute scans by the user performing scan data storage operations or by the scan processing unit 12 referring to the scan mode information. If it is found, the scan data temporarily stored in the scan data holding unit 136 is stored in the image data storage unit 23. At this time, the scan data and sequential identification information are stored in association with each other. The sequential identification information only needs to uniquely identify data stored in the image data storage unit 23, and examples thereof include storage date and time.

  If the scan data is temporarily stored in the scan data holding unit 136 in a state of being associated with each other, the scan data is stored in the image data storage unit 23 in a state of being associated with the scan data.

  The scan data storage operation includes, for example, a user operation on a menu item associated with scan data storage processing. In this case, when the menu display control unit 11 receives the coordinate data corresponding to the position where the menu item is displayed, it can be seen that the user has performed an operation of storing the scan data.

  Next, the scan attribute information setting unit 138 uses an input interface (software keyboard) for updating various types of information stored in the scan attribute information storage unit 22 triggered by the user performing a scan attribute information setting operation. Etc.) are displayed on the liquid crystal panel 301 with a built-in sensor. The scan attribute information setting operation includes, for example, a user operation on a menu item associated with scan attribute information setting processing. In this case, when the menu display control unit 11 receives the coordinate data corresponding to the position where the menu item is displayed, it can be seen that the user has performed the setting operation of the scan attribute information.

  Then, the scan attribute information setting unit 138 changes various information stored in the scan attribute information storage unit 22 based on the contents input by the user through the input interface. When the input interface is a software keyboard, the user input content can be obtained by determining the key displayed at the position indicated by the received coordinate data.

  The scan processing unit 12 refers to the scan mode information stored in the scan mode storage unit 223, and when it is stored that the scan is continuously executed, a predetermined user operation (for example, data display) / Opening / closing operation of the housing of the sensor device 100) is used as a trigger to continuously scan. That is, the set of the scan preprocessing unit 121 and the target object scan execution unit 131 is activated continuously.

  Next, the image display processing unit 18 displays the image data stored in the image data storage unit 23 on the sensor built-in liquid crystal panel 301. Therefore, the image display processing unit 18 refers to the image data storage unit 23 to acquire image data, and displays the acquired image data on the sensor built-in liquid crystal panel 301. At this time, if two pieces of scan data (scan data on the front surface of the object and scan data on the back surface) are stored in association with each other in the image data storage unit 23, the two scan data are displayed simultaneously. May be. Also, only the scan data of the front surfaces of the plurality of objects may be displayed simultaneously, or only the scan data of the back surfaces of the plurality of objects may be displayed simultaneously.

  Further, the image display processing unit 18 displays a menu and buttons for performing processing such as editing on the image data to be displayed on the sensor built-in liquid crystal panel 301.

  Further, the image display processing unit 18 has touched an area on the sensor built-in liquid crystal panel 301A in which no image data, menus, buttons, etc. are displayed, while displaying image data, menus, buttons, etc. May be used as a trigger to refer to the image data storage unit 23 to acquire the image data stored in the order one after the currently displayed image data and display it on the sensor built-in liquid crystal panel 301. Similarly, the image display processing unit 18 touches a region on the sensor built-in liquid crystal panel 301B where no image data, menus, buttons, etc. are displayed, while displaying image data, menus, buttons, etc. With this as a trigger, the image data storage unit 23 may be referred to, and the image data stored in the order immediately before the currently displayed image data may be acquired and displayed on the sensor built-in liquid crystal panel 301.

  Next, the power management unit 19 is driven by the detection image scanning unit 126, the hinge unit switch 904, and the like, and turns on or off the power of the data display / sensor device 100. When the power is turned on, power is supplied to all functions provided in the data display / sensor device 100. On the other hand, when the power is turned off, power is not supplied to all the functions of the data display / sensor device 100.

  In the above description, it is described that user input is performed by a user operating a menu or button displayed on the sensor built-in liquid crystal panel 301. However, the operation unit 903 (switch, remote control, mouse, keyboard, etc.) User input may be performed by a user operating the input device.

(Operation example of the main control unit)
Next, referring to FIG. 10 again, an operation example of the main control unit 800 will be described while touching commands transmitted from the main control unit 800 to the data processing unit 700. Here, an operation example of the main control unit 800 when performing double-sided scanning will be described. In addition, “upper screen” and “lower screen” are set in the “scan execution surface” of the scan pattern storage unit 222, and the scan data obtained as a result of scanning on the upper screen is displayed on the upper screen, It is assumed that the “scan data display surface” of the scan pattern storage unit 222 is set so that scan data obtained as a result of scanning on the lower screen is displayed on the lower screen. The message presented to the user is displayed on the upper screen. In addition, it is assumed that the process associated with the detection image J1 is a scan process for an object.

  First, FIG. 10A is a diagram showing a state in which the placement area R1 is displayed on the lower screen and the message M1 is displayed on the upper screen. In order to perform the display, first, the placement area information presentation unit 122 generates display data including a predetermined pattern image for representing the placement area R1, and transmits the generated display data to the data processing unit 700. To do. In addition, the placement area information presentation unit 122 transmits a command specifying the lower screen (“010”) as the value of the “display panel” field to the data processing unit 700.

  Further, the placement area information presentation unit 122 generates display data including the message M1 and transmits the generated display data to the data processing unit 700. In addition, the placement area information presentation unit 122 transmits a command specifying the upper screen (“001”) as the value of the “display panel” field to the data processing unit 700.

  Next, FIG. 10B is a diagram illustrating a state in which the document P2, which is the object, is placed. Here, when the data display / sensor device 100 detects that the document P2 is placed in the placement area R1, the data display / sensor device 100 displays the button BT1 that can be operated by the user on the sensor built-in liquid crystal panel 301B and the sensor built-in liquid crystal panel 301A. Displays a new message M2.

  First, in order to perform the above detection, the object presence / absence determination unit 123 designates “event” or “all” in the “data acquisition timing” field, and “coordinates” or “partial image” in the “data type” field. A command designating “” is transmitted to the data processing unit 700. When both coordinate data and partial image data (including coordinate data in a partial image) are to be acquired, specify both “coordinate” and “partial image” in the “data type” field (set “011”). specify).

  When the object is placed on the sensor built-in liquid crystal panel 301, the optical sensor circuit 32 detects at least one of a shadow image and a reflected image of the object. As a result, the data processing unit 700 detects the sensor control unit. A change occurs in the image data received from 602.

  Therefore, the data processing unit 700 transmits coordinate data or partial image data (including coordinate data in the partial image) to the target presence / absence determination unit 123 as a response to the above-described command.

  When the object presence / absence determination unit 123 receives coordinate data or partial image data indicating the region of the object from the data processing unit 700, the partial image data is located in the placement region. By examining whether or not there is an object, it can be determined whether or not the object has been placed in the placement area.

  Next, when it is determined that the object is placed in the placement region R1, the object presence / absence determination unit 123 generates display data including the message M2, and transmits the generated display data to the data processing unit 700. . In addition, the object presence / absence determining unit 123 transmits a command specifying the upper screen as the value of the “display panel” field to the data processing unit 700.

  Further, when the object presence / absence determining unit 123 determines that the object is placed in the placement region R1, the detection image display processing unit 124 adds to the predetermined pattern image for representing the placement region R1. Display data including the button BT1 is generated, and the generated display data is transmitted to the data processing unit 700. At the same time, the object presence / absence determining unit 123 transmits a command specifying the lower screen as the value of the “display panel” field to the data processing unit 700.

  Next, FIG.10 (c) is a figure which shows the mode after a user touches button BT1 in the state shown in FIG.10 (b).

  In order to detect that the user has touched the button BT1, the detection image display processing unit 124 designates “event” or “all” in the “data acquisition timing” field, and sets “data type” field to “ A command specifying “coordinate” or “partial image” is transmitted to the data processing unit 700. If both coordinate data and partial image data (including coordinate data in a partial image) are desired, both “coordinate” and “partial image” are specified in the “data type” field.

  When the user touches the button BT1, the optical sensor circuit 32 of the sensor built-in liquid crystal panel 301B detects at least one of a shadow image and a reflection image of a finger, a pen, etc. touched by the user. The image data received by the unit 700 from the sensor control unit 602 changes.

  Therefore, the data processing unit 700 transmits coordinate data or partial image data (including coordinate data in the partial image) to the detection image display processing unit 124 as a response to the above-described command.

  Then, the detection image display processing unit 124 checks whether the coordinate data received from the data processing unit 700 or the partial image data indicating the region of the object is located within the region of the button BT1. Thus, it can be determined whether or not the user has touched the button BT1.

  Next, when it is detected that the user touches the button BT1, the detection image display processing unit 124 generates display data including the detection image J1 in order to display the detection image J1 on the sensor built-in liquid crystal panel 301B. Then, the generated display data is transmitted to the data processing unit 700. At the same time, the detection image display processing unit 124 transmits a command specifying the lower screen as the value of the “display panel” field to the data processing unit 700.

  After displaying the detection image J1, the detection image scanning unit 126 causes the sensor built-in liquid crystal panel 301B to perform a scan in order to determine whether or not the housing of the data display / sensor device 100 is closed. . For this purpose, a command in which “event” or “all” is specified in the “data acquisition timing” field and “whole image” or “partial image” is specified in the “data type” field is transmitted to the data processing unit 700. To do. If it is desired to acquire both the entire image data and the partial image data, both “entire image” and “partial image” are designated in the “data type” field.

  Next, FIG. 10D is a diagram showing a state in which the housing of the data display / sensor device 100 is closed. When the housing of the data display / sensor device 100 is closed, the optical sensor circuit 32 of the sensor built-in liquid crystal panel 301B detects at least one of a shadow image and a reflected image of the image displayed on the sensor built-in liquid crystal panel 301A. As a result, the image data received by the data processing unit 700 from the sensor control unit 602 changes.

  Therefore, the data processing unit 700 transmits the entire image data or the partial image data (including the coordinate data in the partial image) to the detection image scanning unit 126 as a response to the above-described command.

  Then, the detection image scanning unit 126 checks whether the entire image data or the partial image data received from the data processing unit 700 includes an area corresponding to the detection image.

  When an area corresponding to the detection image is included, the object scan execution unit 131 is driven so that the scan execution instruction unit 132 instructs the data processing unit 700 to execute double-sided scanning. A command specifying the “scan method” field and the “scan panel” field is generated, and the generated command is transmitted to the data processing unit 700. Note that the scan execution instructing unit 132 specifies the value of the “scan panel” field based on the scan execution surface information included in the scan pattern information read from the scan pattern storage unit 222. Here, “011” indicating “upper screen” and “lower screen” is designated. Further, “reflection” is designated in the “scan method” field.

  Further, the scan execution instructing unit 132 designates “sense” in the “data acquisition timing” field of the command and acquires “data type” in order to acquire the entire image data as the scan execution result from the data processing unit 700. Specify "Entire image" in the field. Accordingly, the entire image data obtained from the optical sensor circuit 32 via the sensor control unit 602 can be acquired from the data processing unit 700 as a result of scanning. Here, it is possible to acquire image data obtained as a result of scanning on the upper screen and image data obtained as a result of scanning on the lower screen.

  Note that the scan data generation unit 133 extracts scan data from the acquired entire image data. Here, the scan data SCD1 of the front surface of the document P2 is extracted from the entire image data obtained by scanning the upper screen, and the back surface of the document P2 is scanned from the entire image data obtained by scanning the lower screen. Data SCD2 is extracted.

  Next, FIG. 10E is a diagram illustrating a state in which the housing of the data display / sensor device 100 is opened after the scan is executed. Here, the message M3 is displayed on the upper screen, the scan data SCD1 is displayed on the upper screen, and the scan data SCD2 is displayed on the lower screen.

  When the housing is opened, the scan data display instruction unit 135 generates display data including the message M3, and transmits the generated display data to the data processing unit 700. At the same time, the scan execution instructing unit 132 transmits a command specifying the upper screen as the value of the “display panel” field to the data processing unit 700.

  Further, the scan data display instruction unit 135 generates display data including the scan data SCD1, and transmits the generated display data to the data processing unit 700. At the same time, the scan data display instruction unit 135 transmits a command designating the upper screen as the value of the “display panel” field to the data processing unit 700. At this time, the scan data display instruction unit 135 refers to the “scan data display surface” of the scan pattern storage unit 222 and determines the value of the “display panel” field.

  In addition, the scan data display instruction unit 135 generates display data including the scan data SCD2, and transmits the generated display data to the data processing unit 700. In addition, the scan data display instruction unit 135 transmits a command specifying the lower screen as the value of the “display panel” field to the data processing unit 700. At this time, the scan data display instruction unit 135 refers to the “scan data display surface” of the scan pattern storage unit 222 and determines the value of the “display panel” field.

(Processing of data display / sensor device)
Next, the flow of processing when performing scan processing in the data display / sensor device 100 will be described with reference to FIGS. First, FIG. 11 is a flowchart showing a flow of processing in which the user selects a menu in the data display / sensor device 100 immediately after the data display / sensor device 100 is activated.

  First, when the user opens the housing of the data display / sensor device 100 while the power of the data display / sensor device 100 is off (step S11), for example, the hinge switch 904 indicates that the housing has opened. Is detected, and the data display / sensor device 100 is turned on (step S12). The power may be turned on by other operations.

  Then, the menu display control unit 11 displays a user selectable menu on at least one sensor-embedded liquid crystal panel 301 (step S13).

  Then, the menu operation of the user is accepted (step S14), and processing corresponding to the menu operation is performed. Here, a flow is described in which it is assumed that the user selects one of the menu items of “image scan”, “image display”, and “address book display” from the displayed menu. The process flow when “image scan” is selected will be described later. If “display address book” is selected, the address book stored in the storage unit 901 is displayed (step S15). If “display image” is selected, the image data stored in the storage unit 901 is displayed. Although displayed (step S16), the flow of the subsequent processing is not an essential part of the invention, and a description thereof will be omitted.

  Next, FIG. 12 is a flowchart showing a flow of processing for scanning an object by reading a detection image in the data display / sensor device 100. When “image scan” is selected in step S14 in the flowchart shown in FIG. 11, the process shown in the flowchart shown in FIG. 12 is started.

  First, the placement area information presentation unit 122 displays a message prompting the placement area and the object to be placed on the sensor built-in liquid crystal panel 301 (step S21).

  Next, when the user places an object in the placement area (step S22), the object presence / absence determination unit 123 determines that the object has been placed in the placement area (YES in step S23).

  Then, the detection image display processing unit 124 displays a button for receiving an instruction to display the detection image on the sensor-equipped liquid crystal panel 301 (step S24).

  Next, when the user instructs to display the detection image (for example, when the button is touched) (step S25), the detection image display processing unit 124 detects that the detection image display instruction has been issued. (YES in step S26), an image for detection is displayed on the sensor built-in liquid crystal panel 301 (step S27).

  Next, when the user can read the detection image (YES in step S29) by closing the housing of the data display / sensor device 100 (step S28), the detection image scanning unit 126 Driving the object scan execution unit 131 causes the sensor built-in liquid crystal panel 301 to execute a scan (step S30).

  That is, the data processing unit 700 causes the optical sensor circuit 32 to execute scanning via the sensor control unit 602, and the data processing unit 700 transmits image data obtained as a result to the object scan execution unit 131.

  Then, the scan data generation unit 133 extracts scan data from the image data. Then, the scan data generation unit 133 primarily holds the scan data in the scan data holding unit 136 (step S31).

  Thereafter, the object scan execution unit 131 notifies the user that the scan has been completed (step S32). Here, predetermined audio data is output to the audio output unit 908 and notified to the user by audio.

  When the user opens the housing of the data display / sensor device 100 (step S33), the scan data display instruction unit 135 displays a message to notify the user that the scan is completed, and the sensor built-in liquid crystal panel 301. (Step S34). Then, the scan data display instruction unit 135 displays the scan data held in the scan data holding unit 136 (step S35). As a result, the user can check the scan data on the sensor built-in liquid crystal panel 301.

  Next, when the scan processing unit 12 finds that the scan is continuously executed by referring to the scan mode information stored in the scan mode storage unit 223 (YES in step 36), the scan data is stored. The unit 137 stores the scan data temporarily stored in the scan data holding unit 136 in the image data storage unit 23 (step S37). If the scan data is temporarily stored in the scan data holding unit 136 in a state where the scan data is associated, the scan data is stored in the image data storage unit 23 while being associated.

  Subsequently, in order to perform a new scan, a message prompting the user to place the placement area and the object is displayed on the sensor-equipped liquid crystal panel 301 (step S21).

  On the other hand, if it is found that the scan is not continuously executed (NO in step S36), a user operation is accepted (step S37), and a process corresponding to the accepted operation is performed. For example, a flow is described assuming that the user selects one of the menu items “display position change”, “rotate”, “save”, and “character recognition” from the displayed menu. Yes.

  When “display position change” or “rotation” is selected, the process of changing the display position of the scan data (step S39) and the process of displaying the scan data by rotating it to the left by 90 degrees, for example (step S39) S40) is performed, and the user's menu operation is accepted again (step S38).

  If “save” is selected, the scan data storage unit 137 stores the scan data temporarily stored in the scan data holding unit 136 in the image data storage unit 23 (step S41), and the process ends. .

  If “character recognition” is selected, a character recognition process is executed on the displayed scan data (step S42). Since the character recognition process is not an essential part of the invention, a detailed description is omitted here. However, in the character recognition process, characters included in the scan data are recognized, character data is generated, and the generated character data is generated. Is stored in the storage unit 901. In addition, the character data may be stored in the storage unit 901 in a data structure format used in an application such as an address book that manages names, telephone numbers, and addresses in association with each other.

  If the detection image cannot be read in step S29 (NO in step S29), when the user opens the housing of the data display / sensor device 100 (step S43), for example, a scan is executed. A message indicating that it has not been displayed is displayed on the sensor built-in liquid crystal panel 301 (step S44).

(Modification)
In the above description, when the detection image display processing unit 124 displays the detection image on the sensor built-in liquid crystal panel 301, the detection is triggered by the user touching a button for receiving an instruction to display the detection image. Although an example in which an image is displayed has been described, the present invention is not limited to this. A button for accepting an instruction to display a detection image may not be displayed, and the detection image may be displayed simply by triggering that the user touches a predetermined area on the sensor built-in liquid crystal panel 301.

  An example in which a button for receiving a detection image display instruction is not displayed using the data display / sensor device 100 will be described with reference to FIGS. 13 to 18.

  FIG. 13 is a diagram illustrating a state in which a business card P3, which is an object, is placed in a placement region R1 displayed on the sensor built-in liquid crystal panel 301B on the lower screen in a state where the surface is visible to the user. In addition, on the surface of the business card P3, characters "Nara Prefecture Nanda" are written.

  In FIG. 13, in order to inform the user of the operation method, a message M5 prompting the user to touch the sensor built-in liquid crystal panel 301 after placing the object is displayed.

  Next, FIG. 14 shows a state in which the user touches the sensor built-in liquid crystal panel 301B on the lower screen with the finger F1 to display the detection image J1 on the sensor built-in liquid crystal panel 301B in the lower screen in the state of FIG. FIG. As described above, the detection image may be displayed at a position where the user touches the sensor-equipped liquid crystal panel 301 without displaying the button for receiving the detection image display instruction.

  From this state, when the user closes the housing of the data display / sensor device 100, the detection image J1 can be read as a result of scanning with the sensor built-in liquid crystal panel 301A on the upper screen. The scan execution unit 131 scans the business card P3 on the upper screen.

  Next, FIG. 15 is a diagram illustrating a state in which the user opens the housing of the data display / sensor device 100 after executing the scan. As shown in the figure, the scan data SCD3 of the surface of the business card P3 scanned on the upper screen by the object scan execution unit 131 is displayed on the sensor built-in liquid crystal panel 301B on the lower screen.

  In this example, scan data SCD3 is displayed instead of the detection image J1 at the position where the detection image J1 was displayed. As described above, the scan data may be displayed at the position where the detection image is displayed regardless of the scan data display surface information stored in the scan pattern storage unit 222. Accordingly, as shown in FIG. 16, when the user touches the upper sensor liquid crystal panel 301A on the upper screen to display the detection image J1 on the upper sensor liquid crystal panel 301A, the business card P3 is scanned. As shown in FIG. 17, the scan data SCD3 obtained as a result may be displayed at the position where the detection image J1 was displayed on the sensor built-in liquid crystal panel 301A on the upper screen.

  In FIG. 18, the user does not display the detection image J1 (without touching the sensor-equipped liquid crystal panel 301 with the finger F1), and then closes the housing of the data display / sensor device 100, and then the data display / It is a figure which shows a mode that the housing | casing of the sensor apparatus 100 was opened. In this case, since the detection image J1 is not displayed, the detection image J1 cannot be read as a result of the scan performed by the sensor built-in liquid crystal panel 301. Therefore, the object scan execution unit 131 does not execute the scan. Therefore, in this case, since the scan data SCD3 cannot be obtained, the scan data SCD3 is not displayed on the sensor built-in liquid crystal panel 301 as illustrated. In this example, a message M6 for notifying the user that the scan has not been executed is displayed.

[Embodiment 2]
Next, a mode in which the data display / sensor device 100 is turned off by reading the detection image will be described. In the present embodiment, a case where the detection image is displayed as an image showing the placement area of the target object will be described as an example. This embodiment will be described below with reference to FIGS. 19 and 20.

  For convenience of explanation, members having the same functions as those shown in the first embodiment are denoted by the same reference numerals, and description thereof is omitted. Further, the description of the same processing as that described in Embodiment 1 is omitted.

(How to turn off the power of the data display / sensor device 100 by reading the detection image)
First, a typical example of processing for turning off the power of the data display / sensor device 100 by reading a detection image will be described with reference to FIGS. 19 (a) to 19 (c).

  First, FIG. 19A is a diagram showing a state in which a message prompting the user to prepare for scanning is displayed, as in FIG. 10A. A detection image J2 is displayed as an image indicating the placement area of the object to be scanned on the lower-sensor liquid crystal panel 301B, and a message M1 to the user is displayed on the upper-sensor liquid crystal panel 301A. Yes.

  Next, FIG. 19B is a diagram illustrating a state in which the user closes the housing of the data display / sensor device 100 without placing an object in the state illustrated in FIG. At this time, the sensor built-in liquid crystal panel 301A performs scanning in order to read the detection image J2. When it is found that the image data obtained as a result of the scan includes an area corresponding to the detection image J2, the data display / sensor device 100 performs processing associated with the detection image J2. Turn off the power.

  FIG. 19C is a diagram showing a state where the housing of the data display / sensor device 100 is opened after the scan is executed in FIG. As described above, since the power of the data display / sensor device 100 is turned off, nothing is displayed on the sensor built-in liquid crystal panel 301 as illustrated.

  When the user places an object on the detection image J2 and closes the housing of the data display / sensor device 100, the image data obtained as a result of scanning with the sensor built-in liquid crystal panel 301A is displayed. Since it cannot be detected that the corresponding area is included in the detection image J2, the data display / sensor device 100 does not turn off the power.

(Configuration of main control unit 800 and storage unit 901)
The configurations of the main control unit 800 and the storage unit 901 of the data display / sensor device 100 according to the present embodiment are the same as those described in the first embodiment except for the placement area information presentation unit 122.

  In this embodiment, since the detection image is used as an image indicating the placement area of the target object, the placement area information presentation unit 122 according to this embodiment receives the reference target from the scan pattern storage unit 222. The placement area information included in the scan pattern information is read, and a predetermined detection image to be displayed in the area indicated by the placement area information is read from the detection image storage unit 24, and the sensor built-in liquid crystal panel on the lower screen In 301, the read detection image is displayed in the area indicated by the read placement area information.

  It should be noted that processing for turning off the power of the data display / sensor device 100 is associated with the detection image.

(Processing of data display / sensor device)
Next, the flow of processing when the power is turned off by reading the detection image in the data display / sensor device 100 according to the present embodiment will be described with reference to FIG. When “image scan” is selected in step S14 in the flowchart shown in FIG. 11, the processing shown in the flowchart shown in FIG. 20 is started.

  First, the placement area information presentation unit 122 displays a message for prompting the placement area and the object to be placed on the sensor built-in liquid crystal panel 301 (step S51). At this time, the detection image read from the detection image storage unit 24 is displayed in the placement area. In this case, the detection image is assumed to be a predetermined pattern image, for example.

  Next, when the user closes the housing of the data display / sensor device 100 (step S52), the detection image scanning unit 126 scans with the sensor built-in liquid crystal panel 301 on the side where the detection image is not displayed. Is executed. If the image can be read (YES in step S53), and the read image is the detection image (YES in step S54), the detection image scanning unit 126 performs a predetermined process. Since the error voice is output (step S55) and the process associated with the detection image is a process of turning off the power of the data display / sensor device 100 (YES in step 56), power management The power of the data display / sensor device 100 is turned off by driving the unit 19 (step S57).

  If an image that is not the detection image can be read (NO in step S54), for example, the process proceeds to step S31 shown in FIG. If the image cannot be read (NO in step S53), the process returns to step S53.

[Modification]
In each of the above-described embodiments, the flow of executing the processing associated with the detection image by reading the detection image has been described. However, the user moves the sensor built-in liquid crystal panel 301 with a finger, a pen, or the like. When the trace is marked, and the marking can be read by the sensor built-in liquid crystal panel 301 on the side where the marking is not performed, a predetermined process associated with the marking may be executed.

  The predetermined processing includes, for example, (1) scan processing of an object, (2) scan of the object, mask processing of image data obtained as a result of scanning, and (3) scan of the object. And processing for adding information to scan data.

  In the above (1), double-sided scanning / single-sided scanning of an object, scanning only of characters other than a specific color of the object, and the like may be designated. The above (2) includes, for example, a process of generating image data in which only the marked range is masked out of image data obtained as a result of scanning an object. In (3) above, for example, (a) processing for adding information for identifying a category (for example, “work” or “private”) according to the marking in order to classify scan data of an object (B) A process of adding the marking as a memo or graffiti to the scan data of the object (for example, a process of combining the scan data of the object and the marking by general image processing).

  The user may select (1) which predetermined process is to be executed in advance and then perform marking by tracing the sensor built-in liquid crystal panel 301 with a finger, a pen, or the like. Marking may be performed by tracing the sensor-equipped liquid crystal panel 301 with a finger or a pen after the user preselects the associated marking type (for example, color, shape, line thickness, etc.). . The selection of the predetermined process in the above (1) and the selection of the marking type in the above (2) may be performed by, for example, the user touching a menu displayed by the menu display control unit 11 or the like. The association between the marking type and the predetermined process may be set in advance at the time of factory shipment or may be set by the user.

  Further, only when a predetermined number of markings of the same type can be read, the process associated with the marking may be executed. Thereby, the misuse when a user marks by mistake can be prevented.

(Configuration of main control unit 800 and storage unit 901)
Configuration of main control unit 800 and storage unit 901 of data display / sensor device 100 when a user reads a marking of a trace traced with a finger or a pen and executes a predetermined process associated with the type of marking Is the same as the configuration described in the first embodiment except for the detection image display processing unit 124 and the object scan execution unit 131.

  However, the detection image display processing unit 124 acquires coordinate data of a position where the user traces the sensor built-in liquid crystal panel 301 with a finger or a pen, and the marking marking set in advance at the position indicated by the coordinate data. A predetermined image corresponding to the type is displayed.

  In addition, the object scan execution unit 131 may, for example, (1) scan the object, (2) mask the image data obtained as a result of scanning the object, Processing to add information to the scan data of the object is performed. In particular, in the case of (2) above, the scan data generation unit 133 generates image data obtained by masking only the marked range from the image data obtained as a result of scanning the object acquired from the data processing unit 700. . In the case of (3) above, the scan data generation unit 133 performs a process of adding data indicating a category to the scan data or combining the marking with the scan data.

(Process flow)
With reference to FIG. 21, the flow of processing when executing a predetermined processing associated with the type of marking by reading the marking of the trace traced by the user with a finger or a pen will be described. Here, a case will be described in which marking is performed by the user selecting in advance which predetermined process is to be executed, and tracing the sensor-equipped liquid crystal panel 301 with a finger, a pen, or the like. If “image scan” is selected in step S14 in the flowchart shown in FIG. 11, the process shown in the flowchart shown in FIG. 21 is started.

  Then, after determining in step S23 that the object has been placed on the placement area, if the user traces the sensor built-in liquid crystal panel 301 with a finger or a pen (step S61), the detection image display processing unit 124 is displayed. However, a predetermined image as a marking (hereinafter referred to as a marking image) is displayed at the traced coordinate position (step S62).

  Next, when the user closes the housing of the data display / sensor device 100 (step S63), the detection image scanning unit 126 scans with the sensor built-in liquid crystal panel 301 on the side where the marking image is not displayed. Let it run. If the marking image can be read (YES in step S64), the detection image scanning unit 126 executes a predetermined process corresponding to the marking (step S65).

  If the predetermined process is an object scan process, the object scan execution unit 131 executes a scan such as a double-sided scan / single-sided scan of the object, or a scan of only characters other than a specific color of the object (for example). Step S66).

  If the predetermined process is a masking process of image data obtained as a result of scanning the object, the object scan execution unit 131 causes the sensor built-in liquid crystal panel 301 to perform the scan (step S67), and the scan The data generation unit 133 generates scan data that masks a portion corresponding to the marking image in the image data acquired from the data processing unit 700 (step S68).

  If the predetermined process is a process of adding information to the scan data of the object, the object scan execution unit 131 causes the sensor built-in liquid crystal panel 301 to perform a scan (step S69), and the scan data generation unit 133 generates scan data in which the marking image is combined with the image data acquired from the data processing unit 700 (step S70).

(Additional notes)
In the above description, one detection image is displayed on the sensor built-in liquid crystal panel 301, and the detection image is read by the sensor built-in liquid crystal panel 301 on the side where the detection image is not displayed. In the above description, the predetermined process associated with the image is executed. However, a plurality of predetermined detection images are displayed on the sensor built-in liquid crystal panel 301, and the plurality of detection images are read to thereby detect the detection. You may make it perform the predetermined process matched with the image. Thereby, for example, when the predetermined process is a scan of an object, the scan of the object is not executed unless the user displays only one detection image. That is, since the scan of the object is not executed unless the user consciously displays a plurality of predetermined detection images, it is possible to prevent the user from executing the predetermined process by mistake.

  In the above description, the configuration including the data processing unit 700 and the circuit control unit 600 has been described as the configuration of the data display / sensor device 100. However, the configuration is not limited to this configuration. For example, the main control unit 800 may directly control the circuit control unit 600, and the main control unit 800 may generate whole image data / partial image data / coordinate data. The main control unit 800 may directly control the display / light sensor unit 300.

  Finally, the main control unit 800, the data processing unit 700, and the circuit control unit 600 of the data display / sensor device 100 may be configured by hardware logic, or realized by software using a CPU as follows. Also good.

  That is, the data display / sensor device 100 includes a CPU (central processing unit) that executes instructions of a control program for realizing each function, a ROM (read only memory) that stores the program, and a RAM (random access) that expands the program. memory), a storage device (recording medium) such as a memory for storing the program and various data. An object of the present invention is a record in which a program code (execution format program, intermediate code program, source program) of a control program for the data display / sensor device 100, which is software for realizing the functions described above, is recorded so as to be readable by a computer. This can also be achieved by supplying a medium to the data display / sensor device 100 and reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU).

  Examples of the recording medium include tapes such as magnetic tapes and cassette tapes, magnetic disks such as floppy (registered trademark) disks / hard disks, and disks including optical disks such as CD-ROM / MO / MD / DVD / CD-R. Card system such as IC card, IC card (including memory card) / optical card, or semiconductor memory system such as mask ROM / EPROM / EEPROM / flash ROM.

  Further, the data display / sensor device 100 may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available. Also, the transmission medium constituting the communication network is not particularly limited. For example, even in the case of wired such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line, etc., infrared rays such as IrDA and remote control, Bluetooth ( (Registered trademark), IEEE802.11 radio, HDR, mobile phone network, satellite line, terrestrial digital network, and the like can also be used. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

  The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention.

  The present invention can be applied to an apparatus that performs data display processing and scan processing. In particular, the present invention can be suitably applied to a data display / sensor device having a housing in which two liquid crystal panels with built-in sensors face each other.

It is a block diagram which shows the detailed structure of the data display / sensor apparatus which concerns on one Embodiment of this invention. It is a schematic diagram which shows the cross section of the liquid crystal panel with a built-in sensor with which the data display / sensor apparatus which concerns on one Embodiment of this invention is provided. FIG. 3A is a schematic diagram showing a state in which a position touched by a user is detected by detecting a reflected image on a sensor built-in liquid crystal panel included in the data display / sensor device according to the embodiment of the present invention. is there. FIG. 3B is a schematic diagram illustrating a state in which a position touched by the user is detected by detecting a shadow image on the sensor built-in liquid crystal panel included in the data display / sensor device according to the embodiment of the present invention. . It is a block diagram which shows the principal part structure of the data display / sensor apparatus which concerns on one Embodiment of this invention. It is a schematic diagram which shows an example of the frame structure of the command used with the data display / sensor apparatus which concerns on one Embodiment of this invention. It is explanatory drawing which shows an example of the value which can be designated to each field contained in the command shown in FIG. 5, and its outline. FIG. 7A is a result of scanning the entire sensor-equipped liquid crystal panel when the object is not placed on the sensor-equipped liquid crystal panel in the data display / sensor device according to the embodiment of the present invention. Image data. FIG. 7B shows image data obtained as a result of scanning when the user is touching the sensor-equipped liquid crystal panel with a finger in the data display / sensor device according to the embodiment of the present invention. It is a block diagram which shows the structure of the liquid crystal panel with a sensor with which the data display / sensor apparatus which concerns on one Embodiment of this invention is provided, and the structure of its peripheral circuit. FIG. 9A is an explanatory diagram showing a state in which the housing of the data display / sensor device according to the embodiment of the present invention is opened. FIG. 9B is an explanatory diagram showing a state in which the housing of the data display / sensor device according to the embodiment of the present invention is closed. FIG. 10A is a diagram illustrating a state in which the data display / sensor device according to the embodiment of the present invention displays a message that prompts the user to prepare for scanning in a state where scanning can be performed. It is. FIG. 10B is a diagram illustrating a state in which a document that is an object is placed on the data display / sensor device according to the embodiment of the present invention. FIG. 10C is a diagram illustrating a state in which the user has issued a detection image display instruction (touched a button) from the state illustrated in FIG. FIG. 10D is a diagram illustrating a state in which the housing of the data display / sensor device 100 is closed in the state illustrated in FIG. FIG. 10E is an explanatory diagram showing a state in which the housing of the data display / sensor device is opened after the scan is executed in the state shown in FIG. 6 is a flowchart showing a flow of processing in which a user selects a menu immediately after activation of the data display / sensor device in the data display / sensor device according to the embodiment of the present invention. 6 is a flowchart showing a flow of processing for scanning an object by reading a detection image in the data display / sensor device according to the embodiment of the present invention. It is explanatory drawing which shows a mode that the business card was put in the mounting area | region displayed on the sensor built-in liquid crystal panel of the 2nd display / light sensor part of the data display / sensor apparatus which concerns on one Embodiment of this invention. It is explanatory drawing which shows a mode that the user touched on the liquid crystal panel with a built-in sensor of the 2nd display / light sensor part of the data display / sensor apparatus which concerns on one Embodiment of this invention. It is explanatory drawing which shows a mode that the user opened the housing | casing of the said apparatus after performing the scan of the target object in the data display / sensor apparatus which concerns on one Embodiment of this invention. It is explanatory drawing which shows a mode that the user touched on the liquid crystal panel with a built-in sensor of the 1st display / light sensor part of the data display / sensor apparatus which concerns on one Embodiment of this invention. It is explanatory drawing which shows a mode that the user opened the housing | casing of the said apparatus after performing the scan of the target object in the data display / sensor apparatus which concerns on one Embodiment of this invention. After a user scans an object without touching a finger on the sensor built-in liquid crystal panel of the data display / sensor device of the data display / sensor device according to an embodiment of the present invention, It is explanatory drawing which shows a mode that the housing | casing was opened. FIG. 19A shows a placement area and a message prompting the user to prepare for scanning in a state where the data display / sensor device according to another embodiment of the present invention can execute scanning. It is a figure which shows a mode that is doing. FIG. 19B is a diagram illustrating a state in which the housing of the data display / sensor device is closed from the state illustrated in FIG. FIG. 19C is an explanatory diagram showing a state in which the housing of the data display / sensor device is opened after the power is turned off in the state shown in FIG. 6 is a flowchart showing a flow of processing for turning off a power supply by reading a detection image in a data display / sensor device according to another embodiment of the present invention. In a data display / sensor device according to another embodiment of the present invention, a predetermined process associated with the type of marking is executed by reading a marking of a trace traced by a user with a finger or a pen. It is a flowchart which shows the flow of the process at the time.

Explanation of symbols

24 Image storage unit for detection 19 Power supply management unit (processing execution means)
100 Data display / sensor device (image display / image detection device)
123 Object presence / absence determination unit 124 Image display processing unit for detection (image display instruction unit for detection)
126 Image scanning unit for detection (processing execution means)
131 Object scan execution unit (process execution means)
132 Scan execution instruction unit 133 Scan data generation unit 135 Scan data display instruction unit 300 Display / light sensor unit 301 Sensor built-in liquid crystal panel (first planar member, second planar member)
301A Liquid crystal panel with built-in sensor (first planar member or second planar member)
301B Liquid crystal panel with built-in sensor (first planar member or second planar member)
J1 Image for detection J2 Image for detection P1 Document (object)
P2 Document (object)
P3 Business card (object)
R1 placement area

Claims (6)

A housing that can be folded in half;
The two inner surfaces facing each other in a folded state of the casing include a first planar member that displays an image on one inner surface, and a second planar member that detects a nearby image on the other inner surface, Image display / image detection devices respectively arranged to face each other,
A detection image storage unit that stores a predetermined detection image and information indicating a predetermined process that can be executed by the device in association with each other;
Detection image display instruction means for displaying the detection image on the first planar member;
When the detection image display instruction means detects the detection image displayed on the first planar member on the second planar member in a state where the casing is folded, the detection image is displayed. Image display comprising: process execution means for acquiring information indicating the associated predetermined process from the detection image storage unit and executing the predetermined process indicated by the acquired information / Image detection device.   The predetermined process is a process of detecting an image of an object placed on the first planar member or the second planar member with the second planar member in a state where the casing is folded. The image display / image detection apparatus according to claim 1, wherein: The first planar member further detects a nearby image,
In the predetermined process, the image of the object placed on the first planar member or the second planar member is detected by the first planar member in a state where the casing is folded. The image display / image detection apparatus according to claim 1, wherein the image display / image detection apparatus is a process.   The detection image display instruction means receives a user instruction to display the detection image and displays the detection image on the first planar member in response to the user instruction. The image display / image detection apparatus according to any one of 1 to 3.   The image display / image detection apparatus according to claim 1, wherein the predetermined process is a process of turning off the power of the own apparatus.   6. The image display / image detection apparatus according to claim 1, wherein the detection image display instruction means displays the detection image in a region where an object is placed.

Images