JP5901680B2 - Portable terminal device and display device - Google Patents

Description

  The present invention relates to a portable terminal device that reads information from an image displayed on a display device.

  An image including information such as numerical values may be displayed on a display device used for system monitoring or the like. In reading such information, reading information that changes with the passage of time, that is, so-called dynamic information has various utility values, and thus is particularly necessary.

  In general, such information is acquired when a device connected to be able to communicate with the display device receives image data from the display device. For example, in the system disclosed in Patent Document 1, the display content of the programmable display can be confirmed on the client device via the host computer and the Internet.

  However, in a device that is not communicably connected to the display device, information on an image displayed on the display device cannot be stored as data. Therefore, it is conceivable that information is read and handled as data by capturing an image displayed on the display device and performing image processing on the captured image.

  The position of the image to be image-processed can be specified by an image marker arranged close to the four corners of the display area in the display device, and is acquired together with the image marker by being imaged by the camera. In the image processing, it is possible to estimate the position and inclination (direction) of the acquired image with respect to the camera by analyzing the image marker. A general image marker is formed in a rectangular shape and includes a black frame forming an outer shape and a white area inside the black frame. Since such an image marker is not easily affected by brightness, image processing is easy even if the brightness of the room changes.

  In general, image processing of a read image requires a large amount of calculation processing that cannot be ignored.To reduce this calculation processing, the image of the entire captured image is analyzed to determine the position of the information. It is necessary to reduce the amount of calculation to search for. Moreover, in general, advanced image processing such as analyzing the entire image increases the possibility of erroneous determination as to whether or not the analyzed portion of the image is information, thus reducing erroneous determination. It is desirable to limit the display position of information. With regard to facilitating information search, it is desirable to arrange information efficiently because a display device usually has a plurality of images and the layout of each image is often different.

  In order to arrange information efficiently, for example, a technique described in Patent Document 2 can be used. Patent Document 2 describes that, when the display state of a window is changed in the entire display image, information that is desired to be always displayed is re-displayed at an optimal position in an empty area of the display image at an optimal size. Yes. As a result, information can be displayed at an optimum position in the empty area of the display image.

JP 2001-268106 A (published on September 28, 2001) JP 2004-030296 A (published January 29, 2004)

  However, the method for reading a captured image has the following problems.

  First, in order to extend the image reading distance by the image marker, it is necessary to enlarge the image marker. However, since the size of the image marker is practically about 10 cm square due to restrictions on the arrangement area, the reading distance remains within 1 m. Moreover, since a general image marker is a printed matter, in order to change the information included in the image marker, it is necessary to create and reprint the image marker based on the changed information. For this reason, it is not easy to change the information included in the image marker.

  Further, since the display device cannot recognize that the information is read, there is a need to always display the information. Alternatively, in order for the display device to recognize that the information is read, some device for recognition other than the image processing device is required, which increases the cost.

  When information is always displayed, it is conceivable to display the information in a format such as a barcode or a QR code (registered trademark) in order to facilitate image processing of the information. However, since the displayed information is low in human readability, when information is always displayed by providing a dedicated area for an image for monitoring or operation, the original display function of the image is restricted.

  The present invention has been made to solve the above problems, and an object thereof is to efficiently read information displayed on a display device.

  In order to solve the above-described problem, a mobile terminal device according to the present invention includes a display area that is an image of a display area portion of the display device from an entire image of the imaging device and the display device imaged by the imaging device. In order to capture an image, a marker recognition unit that recognizes a marker that specifies a range of the display region, and a relative position that specifies a relative positional relationship of the display device with respect to the imaging unit based on the recognized marker Based on the specifying means, the image correcting means for correcting the captured display area image based on the relative positional relationship, and the information display code included in the corrected display area image is detected and analyzed to obtain information. And an information reading means for reading.

  In the above configuration, when the marker is recognized from the entire image of the captured display device, the image displayed in the display area is captured. The relative positional relationship (distance, direction, angle, etc.) of the display device with respect to the imaging means is specified by the recognized marker.

  The captured image is corrected based on the above relative positional relationship. When an information display code is detected in the corrected image and the detected information display code is analyzed, information represented by the information display code is read.

  Thus, since the marker is a large image marker that specifies the range of the display area, the image reading distance by the marker can be extended to 1 m or more. Moreover, since the information display code is read from the image displayed on the display device by the mobile terminal device, information can be easily obtained from the information display code included in the image by the mobile terminal device that is not connected to the display device so as to communicate can do.

  The portable terminal device includes storage means for storing arrangement information indicating an arrangementable area in which the information display code can be arranged on the specified display device, and the information reading unit is configured such that the marker is the display device. When the display device specifying information for specifying the display device is included, the display device is specified based on the display device specifying information, and by referring to the disposition information set for the specified display device, It is preferable to detect the information display code.

  In the above configuration, the information display code may be detected by narrowing down to the arrangementable area by referring to the arrangementable information about the specified display device. Thereby, the information display code can be efficiently detected without performing an extra search for detection.

  In the portable terminal device, it is preferable that the display area image includes an image indicating information on the information display code, and the information reading unit detects the information display code based on the information.

  According to said structure, since the detection of an information display code is performed based on the information regarding an information display code, a detection can be performed more efficiently.

  In the portable terminal device, it is preferable that the report reading unit detects the information display code by changing a region in a predetermined order from a region where the information display code is predicted to be arranged in the corrected image.

  According to the above configuration, the information display code is detected in a predetermined order from an area where the information display code is expected to be arranged, for example, a part in contact with the four sides or four corners of the image. Can increase the possibility of detecting.

  The display device according to the present invention includes a display region, a marker provided for the mobile terminal device to identify a range of the display region from an entire image obtained by imaging the entire display device, and the portable device. Image display means for displaying an image provided with an information display code representing predetermined information for reading by the terminal device in the display area, wherein the marker identifies the display device by the portable terminal device Therefore, the display device specifying information provided for the purpose is included.

  According to the above configuration, in combination with the above-described portable terminal device, the reading distance of the image by the marker is extended, and the information included in the captured image in the portable terminal device that is not communicably connected to the display device Information can be easily obtained from the display code.

  In the display device, it is preferable that the image display unit displays the image including position information indicating a display position of the information display code.

  According to said structure, an information display code can be displayed so that a portable terminal device can detect an information display code efficiently.

  The present invention, which is configured as described above, produces an effect that information displayed can be read efficiently.

1 is a block diagram illustrating a configuration of an image reading system according to Embodiments 1 and 2 of the present invention. It is a front view which shows the external appearance structure of the display apparatus in the said image reading system. It is a front view which shows the said display apparatus with which the image marker is not affixed. It is a front view which shows the structure of the said image marker. It is a front view which shows the said display apparatus of the state by which the information which specifies the said display apparatus was provided to the said image marker. It is a front view which shows the external appearance structure of the portable terminal device in the said image reading system. It is a figure which shows an example of the image displayed on the said display apparatus. It is a figure which shows the state by which the information display code was displayed on the image shown in FIG. It is a figure which shows the other state by which the information display code was displayed on the image shown in FIG. It is a figure which shows the display area part of the display apparatus read by the said portable terminal device. It is a figure which shows an example of the image displayed on the display apparatus of the image reading system which concerns on Embodiment 2 of this invention. It is a figure which shows the other example of the image displayed on the display apparatus of the image reading system which concerns on Embodiment 2 of this invention. It is a figure which shows the state by which the information display code was displayed on the image shown in FIG.

[Embodiment 1]
Embodiment 1 of the present invention will be described below with reference to FIGS.

  FIG. 1 is a block diagram illustrating a configuration of an image reading system 10 according to the first and second embodiments. FIG. 2 is a front view showing an external configuration of the display device 1 in the image reading system 10. FIG. 3 is a front view showing the display device 1 on which no image marker is attached. FIG. 4 is a front view showing the configuration of the image marker. FIG. 5 is a front view showing the display device 1 in a state where information for specifying the display device 1 is given to the image marker.

  As shown in FIG. 1, the image reading system 10 includes a display device 1 that displays an image (including both moving images and still images) and a mobile terminal device 2 that images the display device 1. The display device 1 is not connected to the mobile terminal device 2 via a network or the like, and is provided in a state where it cannot communicate with each other.

  First, the external configuration of the display device 1 will be described.

  As shown in FIGS. 2 to 4, the display device 1 includes a housing 11 that is rectangular when viewed from the front, a display panel 12 (display area) that is disposed on the front surface (display surface) of the housing 11, and a housing. An image marker (hereinafter referred to as “marker 13”) attached to the body 11 is provided. When the display device 1 is configured as an operation type display device such as a programmable display capable of touch operation, a touch panel (not shown) is provided on the display panel 12.

  The housing 11 has a frame-like bezel (edge) 11 a provided so as to surround the display panel 12. Further, the marker 13 is disposed so as to surround the display surface of the display panel 12 as shown in FIG. 2 by being attached to almost the entire surface of the bezel 11a shown in FIG. By arranging the marker 13 in this way, the range of the display area is specified as the range of the opening inside the marker 13.

  As shown in FIG. 4, the marker 13 has substantially the same frame shape as the bezel 11a, and has black frame portions 13a and 13b and a white frame portion 13c. The outer black frame portion 13 a is provided on the outer edge portion of the marker 13, and the inner black frame portion 13 b is provided on the inner edge portion of the marker 13. The white frame portion 13c is provided between the outer black frame portion 13a and the inner black frame portion 13b. As described above, the marker 13 is less affected by ambient brightness by combining the outer black frame portion 13a, the inner black frame portion 13b, and the white frame portion 13c. It is easy to be recognized as.

  As shown in FIG. 5, the marker 13 may be provided with a plurality of marks 13 d as information (display device specifying information) for specifying the display device 1. The mark 13d has a black rectangular shape and is formed on the white frame portion 13c with a predetermined interval. The arrangement position and the arrangement interval of the mark 13d are specific to each display device 1, and the display device 1 can be specified by the mark 13d.

  Next, the external configuration of the mobile terminal device 2 will be described.

  As shown in FIG. 6, the mobile terminal device 2 includes a casing 21 that is rectangular when viewed from the front, a display panel 22 provided on the casing 21, a touch panel 23 provided on the display panel 22, and a casing. The camera module 24 (imaging means) provided on the body 21 is provided. The camera module 24 includes an imaging lens that captures external light and an imaging element that converts an image formed on the light receiving surface by the imaging lens into an image signal and outputs the image signal.

  The portable terminal device 2 described above is configured as a tablet terminal or the like, and can perform display and touch operations on the front surface with the display panel 22 and the touch panel 23 and can perform photographing with the camera module 24.

  Furthermore, each system structure of the display apparatus 1 and the portable terminal device 2 is demonstrated.

  As shown in FIG. 1, the display device 1 includes a display control unit 14 that controls image display, a data format conversion unit 15 that displays acquired data as an information display code, and information that is given to the display control unit 14. A memory 16 for storing code display data for displaying a display code is provided.

  The memory 16 stores a plurality of image data. The image represented by the image data is configured by arranging various component images in the background area, and the component image may have the same (or similar) arrangement position, or may be different.

  The data format conversion unit 15 converts various data generated outside or inside the display device 1 into code display data provided as dynamically changing information. The code display data is data visualized as an information display code. The information display code can be recognized as an image marker by the mobile terminal device 2. As the information display code, for example, a code such as a barcode or a QR code (registered trademark) is used.

  The information visualized as the information display code is not limited to a barcode, QR code (registered trademark), or the like, but can be decoded into a form that can be recognized by a person such as letters and numbers in the mobile terminal device 2. Information is desirable.

  The display control unit 14 (image display means) causes the display panel 12 to display an image based on the image data stored in the memory 16. The display control unit 14 has an information display processing unit 14a for displaying the code display data as an information display code.

  The information display processing unit 14a sets an unused area of a predetermined size or more in a background area where no component image is arranged in the image. In addition, the information display processing unit 14 a controls the display panel 12 based on the code display data acquired from the memory 16 so as to display the information display code as visualized information in the unused area.

  On the other hand, the mobile terminal device 2 further includes an image reading unit 25 that reads the above-described information display code, a memory 26 that stores information used for reading the information display code, and a control unit that controls the entire mobile terminal device 2. 27.

  The memory 26 (storage means) stores display device specifying information and image layout information. The display device specifying information is information for specifying each of a plurality of display devices. For example, the display device specifying information is provided as a table in which the ID and model (model number) of the display device 1 are associated with the ID and model number. ing.

  Image layout information is information relating to the layout (layout) of component images in an image. Among all images, images having similar or identical component image layouts are grouped together, and the layout of component images is patterned for each group. Is. Usually, when the display device 1 to be imaged is different, the image data stored in the display device 1 is also different. Therefore, the image layout information is provided according to a plurality of registered display devices 1. Further, the image layout information (arrangeable information) can specify an area (arrangeable area / unused area) where the information display code can be arranged other than the area where the component image is arranged.

  The image reading unit 25 includes a marker recognizing unit 25 a that recognizes the marker 13, a relative position specifying unit 25 b that specifies the relative positional relationship of the display device 1 with respect to the camera module 24, and an image correcting unit that corrects the captured image. 25c and an information reading unit 25d that reads information from the corrected image.

  The marker recognizing unit 25a (marker recognizing means) is displayed on the display panel 12 by recognizing the marker 13 from the entire image of the display device 1 imaged by the camera module 24 using the image marker reading technique. Recognize and capture an image part (display area part).

  The relative position specifying unit 25b (relative position specifying means) estimates the relative positional relationship (distance, direction, angle, etc.) of the display device 1 with respect to the camera module 24 based on the marker 13 recognized by the marker recognition unit 25a. Specified.

  The image correction unit 25c (image correction unit) corrects the captured display area portion to an image viewed from the front based on the relative positional relationship.

  The information reading unit 25d (information reading means) detects an information display code in the captured display area portion. In addition, the information reading unit 25d generates code display data from the detected information display code, and decodes the code display data into original data (data before being converted into code display data by the display device 1). To do.

  The control unit 27 performs display control of the display panel 22, control of each unit based on an input operation by the touch panel 23, and the like. Further, the control unit 27 causes the output unit (the display panel 22 or a speaker (not shown)) to output information represented by the data based on the data decoded by the image reading unit 25, that is, the original data.

  Subsequently, an information display operation and a reading operation by the image reading system 10 configured as described above will be described.

  FIG. 7 is a diagram illustrating an example of an image (image 100) displayed on the display device 1. As illustrated in FIG. FIG. 8 is a diagram illustrating a state in which information is displayed on the image 100. FIG. 9 is a diagram illustrating another state in which information is displayed on the image 100. FIG. 10 is a diagram illustrating a portion of an image read by the mobile terminal device 2.

  First, the display device 1 displays an image 100 as shown in FIG. This image 100 is configured by arranging a plurality of component images 102 in the background area 101. The component image 102 is provided as a component image representing a switch for operation, a component image for numerical display, a component image representing a lamp, and the like.

  The data format conversion unit 15 converts various data generated outside or inside the display device 1 into code display data and stores the data in the memory 16 as information. There are various types of information visualized as information display codes. For example, information visualized as an information display code includes information that only an administrator can understand, maintenance information, and information (translation etc.) that is converted into content. Examples of specific data to be specifically converted include data such as alarm data (error data) and numerical data (such as output data of a measuring instrument).

  The information display processing unit 14a refers to the image data and selects a rectangular unused area 103 having a predetermined size or larger from an area where the part image 102 is not arranged in the background area 101 as a candidate for displaying an information display code. Set as area. In the example shown in FIG. 7, unused areas 103 are set on the left and right sides of the upper end portion and the lower end portion of the background area 101. Since the unused area 103 is set on the image data developed in the display memory (not shown), a two-dot chain line indicating the unused area 103 appears in the actually displayed image 100. Not.

  The information display processing unit 14a considers the size and aspect ratio from the set unused area 103, and the unused area 103 (which is most suitable for displaying an information display code that can be image-processed by the mobile terminal device 2). Specifically, it depends on the display format of the information display code to be displayed. Further, the information display processing unit 14 a controls the display panel 12 based on the code display data acquired from the memory 16 so as to display the information display code in the selected unused area 103.

  As a result, for example, as shown in FIG. 8, in the image 100, the barcode is displayed as the information display code 104 in the unused area 103 having the largest size at the lower right end in the background area 101 shown in FIG. 7.

  In addition, the information display processing unit 14a performs the image processing for confirming whether the displayed image is arranged pixel by pixel without referring to the image data as described above, thereby removing the unused area 103. It may be set. However, in this setting method, since time is required for image processing, it is preferable to set the unused area 103 with reference to image data in order to increase the efficiency of the processing.

  The unused area 103 for displaying the information display code is set by being automatically selected from the plurality of unused areas 103 by the information display processing unit 14a, but may be set in advance by the user. . When the unused area 103 is set by the user, for example, when the user creates image data, the unused area 103 is also set.

  It is also possible to combine the setting of the unused area 103 by the information display processing unit 14a and the setting of the unused area 103 by the user. Specifically, as shown in FIG. 9, the user fixes the position of the unused area 103A for displaying position information. The position information defines the position (coordinates) of the unused area 103B where the information display code is displayed, and is displayed in the unused area 103A as an information display code, for example. The information display processing unit 14a sets an unused area 103B to be displayed as an information display code at the position of the position information set in the unused area 103A.

  As a result, it is determined that the position information of the fixed unused area 103A is read first, and the information display code displayed in the unused area 103B is read based on the obtained display position. Therefore, since the number of position candidates for displaying the information display code is always limited to one, it is not necessary to detect the optimum unused area 103 for displaying the information display code.

  In the above description, an example in which a code indicating position information indicating the position of the unused area 103B is displayed in the unused area 103A has been described, but the information indicated by the code is not limited to the position information described above. . For example, the code displayed in the unused area 103A is, for example, whether the information display code displayed in the unused area 103B is a bar code or a QR code (registered trademark), and the information display code has a square shape. Various types of information, such as whether it is a triangle or a triangle, that is, information about an information display code may be expressed.

  In addition, since the amount of position information is not large, the information display code for displaying the position information can be reduced. Thereby, the freedom degree of arrangement | positioning of the positional information in the image 100 can be raised. In addition, since the image reading unit 25 (described later) in the mobile terminal device 2 fixes and arranges the position information at a position (for example, one of the four corners) where detection of the information display code is started, reading is started. It becomes possible to confirm the position information immediately.

  On the other hand, in the mobile terminal device 2, when the entire display device 1 is imaged by the camera module 24, an entire image (image signal) of the display device 1 output from the camera module 24 is given to the image reading unit 25.

  In the image reading unit 25, the marker recognition unit 25 a recognizes the marker 13 from the entire image of the display device 1. At this time, the marker recognizing unit 25a detects that the pattern is the marker 13 by detecting the patterns of the black frame portions 13a and 13b and the white frame portion 13c of the marker 13. Further, the marker recognizing unit 25 a recognizes the marker 13, and captures the portion of the image displayed in the display area inside the marker 13 as the display area image 200 from the entire image as shown in FIG. 10.

  Based on the size and shape of the rectangular area represented by the marker 13 recognized by the marker recognition unit 25a and the reference information about the size and shape of the marker 13, the relative position specifying unit 25b is connected to the camera module 24 (mobile phone). The relative positional relationship of the display device 1 with respect to the terminal device 2) is estimated. The basic information is stored in advance in the memory 26 or the like. Based on this relative positional relationship, the position (distance from the camera module 24), size, posture (tilt with respect to the camera module 24), and the like of the display device 1 can be specified.

  The image correcting unit 25c corrects the captured display area image 200 into an image viewed from the front based on the relative positional relationship specified by the relative position specifying unit 25b. When the entire image of the display device 1 is taken from an oblique direction, the display area image 200 is distorted. Even if the information reading unit 25d tries to detect the information display code from the display area image 200 having such a distorted shape as described later, it may not be detected correctly depending on the degree of distortion. For this reason, the image correction unit 25c corrects the display area image 200 including the information display code into a correct shape based on the relative positional relationship as described below.

  The image correcting unit 25c corrects the size of the display area image 200 so as to be an appropriate size based on the distance (imaging distance) of the display device 1 with respect to the mobile terminal device 2 in the relative positional relationship. For example, when the display device 1 is imaged from a close position, the large display area image 200 is captured, so the image correction unit 25c generates a correction magnification smaller than 1 according to the imaging distance, and based on the correction magnification. The display area image 200 is corrected so as to be reduced.

  In addition, the image correction unit 25c corrects the inclination of the display area image 200 so as to face the erect direction based on the direction (imaging direction) of the display device 1 with respect to the mobile terminal device 2 in the relative positional relationship. . For example, when the display device 1 is imaged from the lower right side, the display area image 200 is also inclined obliquely according to the imaging direction. Therefore, the image correction unit 25c generates a correction angle according to the imaging direction, and The inclination of the display area image 200 is corrected based on the correction angle.

  Furthermore, the image correction unit 25 c corrects the balance of the overall shape of the display area image 200 based on the angle (imaging angle) of the display device 1 with respect to the mobile terminal device 2. As the angle at which the display device 1 is imaged increases with respect to the front direction, the aspect ratio of the display area image 200 (ratio of the horizontal dimension to the vertical dimension) differs. For example, when the display device 1 is imaged from the upper and lower oblique directions, when the imaging angle is increased, the horizontal dimension is not so short, and the vertical dimension is shortened, so that the aspect ratio is increased. On the other hand, when the display device 1 is imaged from the left and right oblique directions, when the imaging angle is increased, the vertical dimension is not so short, and the horizontal dimension is shortened, so the aspect ratio is small. In any of the above cases, the image correction unit 25c generates a correction ratio according to the imaging angle, and corrects the aspect ratio of the display area image 200 to an appropriate value based on the correction ratio.

  The information reading unit 25d compares the code number decoded from the code represented by the mark 13d with each code number of the display device specifying information in the memory 26, and reads the display device corresponding to the code number that matches the code number. The target display device 1 is specified. The information reading unit 25d refers to the image layout information corresponding to the specified display device 1, and in the display area image 200 corrected as described above, the information display code from the background area where no component image is arranged. Looking for. Regarding the reference to the image layout information, a plurality of pieces of image layout information are referred to one by one until the information display code 201 is detected. For example, when there is first to nth image layout information, first, when the information display code 201 cannot be detected by referring to the first image layout information, the second image layout information is referred to in advance. The image layout information is referred to in a predetermined order. By referring to the image layout information in this way, locations where information display codes are likely to be arranged, that is, unused areas are narrowed down, so that the information display code 201 can be detected quickly.

  Although not shown, when the display area image 200 includes an information display code for displaying the position information, the information reading unit 25d detects the information display code as described above, The position where the display code is displayed can be specified. This eliminates the need for further information display code detection.

  In addition, the information reading unit 25d generates code display data from the detected information display code, analyzes the code display data, and generates original data (data before being converted into code display data by the display device 1). Decrypt into

  The control unit 27 creates display data in the form of text, symbols, and the like based on the code display data and supplies the display data to the display panel 22. Thereby, the information included in the code display data is displayed on the display panel 22 in a form that can be read by the user. For example, information is displayed at a predetermined position in the form of a character string or a number in a display image. Further, the control unit 27 may generate audio data based on the code display data and give it to a speaker (not shown). As a result, information included in the code display data and an alarm generated by the information are notified from the speaker by a message or electronic sound that can be understood by the user.

  In addition to outputting information as described above, the control unit 27 may activate a specific application program installed in the mobile terminal device 2 using the information as a trigger.

  As described above, in the image reading system 10 according to the present embodiment, the marker 13 is provided around the display panel 12 of the display device 1, so that the mobile terminal device 2 can detect the marker 13 from the entire image of the display device 1 captured. Recognize Thereby, the relative positional relationship (distance, direction, angle, etc.) of the display device 1 with respect to the mobile terminal device 2 can be specified. In addition, since the large marker 13 corresponding to the size of the display panel 12 is used, the image reading distance by the marker 13 can be extended to 1 m or more.

  Further, in the image reading system 10, the information display code is read and output from the image displayed on the display device 1 by the mobile terminal device 2. Thereby, information can be easily acquired from the information display code included in the image by the mobile terminal device 2 that is not connected to the display device 1 so as to be communicable.

  In addition, the mobile terminal device 2 provides information about where there is an area where an unused area can be set for all images displayed on the display device 1 instead of detailed information such as image layout information. You may have.

  Further, in the image reading system 10, when each image can be classified into several groups according to the arrangement pattern such as a component image in each image, the mobile terminal device 2 uses the image layout information to display information. Detect code. The display device 1 corresponding to the image layout information is specified by the mark 13d of the marker 13. As a result, it is possible to easily narrow down an area where a part image or the like, which is likely to have an information display code, is not arranged, or to specify one area. Therefore, the information display code can be detected efficiently.

  Further, as shown in FIG. 9, the unused area 103B is confirmed by confirming the display position of the information display code based on the position information (information indicating the position where the information display code is displayed) set in the unused area 103A. It is possible to detect the information display code displayed on the screen. In this case, the information display code can be efficiently detected by arranging the position information at the location where the detection is first performed.

  The information visualized as the information display code may be displayed during the monitoring idle time or may be displayed a predetermined number of times per second. In addition, the information may be displayed as special reset information within a range that does not affect the information. Further, the information may be not only constant information such as an error but also information that changes on a time axis. Alternatively, as the information, not only the same information but also different information can be displayed. For example, the temperature change information of a certain device may be displayed in real time, the temperature information of a certain device may be displayed at one time, and the flow rate information of another device may be displayed at another time. .

[Modification]
In the above-described example, the display device 1 is specified using the marker 13 including the mark 13d as illustrated in FIG. 5 and the image layout information corresponding to the specified display device 1 is referred to efficiently. The information display code can be detected. On the other hand, when the marker 13 that does not include the mark 13d as shown in FIG. 4 is used, the display device 1 cannot be specified, and thus the image layout information cannot be referred to. The code can be searched efficiently. Next, an example will be described.

  As shown in FIG. 10, the information reading unit 25d starts detection of the information display code 201 from the portions and the four corners in contact with the four sides of the captured display area image 200, and continues until the information display code is detected. The detection is continued by changing the region in this order. Specifically, the information display code 201 is searched in the order of the four corners, the part in contact with the four sides, and the other part, but the search may be started from the part in contact with the four sides. In addition, the order is determined in each of the corners and the portions in contact with the four sides. For the four corners, for example, the order of the upper right corner, the upper left corner, the lower left corner, and the lower right corner is determined, but the order may be other than this order. For the parts in contact with the four sides, for example, the order of the part in contact with the upper side, the part in contact with the left side, the part in contact with the lower side, and the part in contact with the right side is determined. Further, in other portions, for example, the search is sequentially performed in the horizontal direction (from the right end to the left end) from the upper portion to the lower portion, but other orders may be used.

  Since it is highly possible that a part image is not arranged at the portions and four corners in contact with the four sides in the display area image 200, it can be predicted that the information display code is arranged. Therefore, the information display code 201 can be detected quickly by searching for the information display code 201 first from the part in contact with the four sides or the four corners.

  As described above, even when the display device 1 cannot be specified, the information display code can be efficiently detected by searching the information display code in order from the part in contact with the four sides and the four corners. In this case, the information display code of the unused area can be easily read without referring to the image layout information, and the image layout information is not necessary, so that the display device 1 need not be specified.

  However, it cannot be said that the information display code can be efficiently detected for the image in which the information display code is arranged in a part in contact with the four sides or a part other than the four corners. On the other hand, specifying the display device 1 using the marker 13 including the mark 103d and referring to the image layout information corresponding to the specified display device 1 is more efficient even in the above-described image. Therefore, it is preferable because the information display code can be detected.

[Embodiment 2]
Embodiment 2 of the present invention will be described below with reference to FIGS. 1, 4, and 11.

  In the present embodiment, components having the same functions as the components in the first embodiment are denoted by the same reference numerals, and the description thereof is omitted.

  FIG. 11 is a diagram illustrating an example of an image displayed on the display device 1 of the image reading system 10 according to the second embodiment.

  In the image reading system 10 according to the first embodiment, the marker 13 includes a mark 13 d that can identify the display device 1. However, when the ID of the display device 1 needs to be changed, it is necessary to create (print) the marker 13 including the mark 13d corresponding to the changed ID and paste it on the bezel 11a.

  Therefore, in the image reading system 10 according to the present embodiment, the display device 1 is equipped with a marker 13 that does not include the mark 13d as shown in FIG. The information display processing unit 14a of the display device 1 displays a marker serving as the mark 13d on the image. Specifically, as shown in FIG. 11, the information display processing unit 14a controls the display panel 12 to display a marker 106 formed in a frame shape on the outer periphery of the image 100A. The marker 106 has a predetermined number of white rectangular marks 106b arranged at a predetermined interval in a black band-shaped base region 106a. That is, the display device 1 can be specified by the preset number and the interval. The marker 106 identifies the inside of the marker 106 as a portion used for display, that is, a display area.

  An information display code can be displayed in the marker 106, or information for specifying the position, shape, and type of the information display code can be displayed.

  Further, in the above embodiment, the display device specifying information for specifying the display device 1 is configured by the number of white rectangular marks 106b and their intervals, but the display device specifying information is not limited to this configuration. . For example, a triangular mark may be used instead of the white rectangular mark 106b, and various shapes of marks can be used. Furthermore, the display device 1 can be specified by the position where the mark is arranged.

  In the present embodiment, the display device 1 is specified by the mobile terminal device 2 as described above, but the other processing by the mobile terminal device 2 is the same as that of the first embodiment. Accordingly, the information display code detection process and the information reading process from the detected information display code are also performed as described in the first embodiment.

[Modification of Embodiments 1 and 2]
Here, a modified example of the first and second embodiments will be described.

  FIG. 12 is a diagram illustrating another example (image 300) of an image displayed on the display device 1. FIG. 13 is a diagram showing a state in which the information display code 304 is displayed on the image 300 shown in FIG.

  In the first embodiment, as illustrated in FIG. 8, the information display code 104 is rectangular. However, the overall shape of the information display code is not limited to a rectangle, and may be another shape.

  For example, in the image 300 shown in FIG. 12, since the component image 302 is arranged up to the vicinity of the four sides of the background area 301, a rectangular unused area cannot be secured. In such a case, the user or the information display processing unit 14a sets a triangular unused area 303 at a vacant place in the background area 301, for example, two corners on the lower side. In the unused area 303, as shown in FIG. 13, a bar code information display code 304 is displayed under the control of the information display processing unit 14a.

[Example of software implementation]
The control blocks (particularly the display control unit 14, the data format conversion unit 15, and the image reading unit 25) of the display device 1 and the portable terminal device 2 are realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like. Alternatively, it may be realized by software using a CPU (Central Processing Unit).

  In the latter case, the display device 1 and the mobile terminal device 2 include a CPU that executes instructions of a program that is software that realizes each function, and a ROM (in which the program and various data are recorded so as to be readable by a computer (or CPU)). A Read Only Memory) or a storage device (these are referred to as “recording media”), a RAM (Random Access Memory) for expanding the program, and the like are provided. And the objective of this invention is achieved when a computer (or CPU) reads the said program from the said recording medium and runs it. As the recording medium, a “non-temporary tangible medium” such as a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. The program may be supplied to the computer via an arbitrary transmission medium (such as a communication network or a broadcast wave) that can transmit the program. The present invention can also be realized in the form of a data signal embedded in a carrier wave in which the program is embodied by electronic transmission.

  The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention.

  The present invention can be used for reading information included in an image displayed on a display device.

DESCRIPTION OF SYMBOLS 1 Display apparatus 2 Portable terminal device 12 Display panel (display area)
13 Marker 14 Display control unit (image display means)
14a Information display processing unit 24 Camera module (imaging means)
25a Marker recognition unit (marker recognition means)
25b Relative position specifying unit (relative position specifying means)
25c Image correction unit (image correction means)
25d Information reading unit (information reading means)
26 Memory (storage means)
100, 100A, 300 Images 103, 303 Unused area 104 Information display code 106 Information display code (marker)
200 Display area image

Claims (3)

Imaging means;
Marker recognition means for recognizing a marker for specifying a range of the display area in order to capture a display area image which is an image of a display area portion in the display apparatus from an entire image of the display apparatus imaged by the imaging means; ,
Relative position specifying means for specifying a relative positional relationship of the display device with respect to the imaging means based on the recognized marker;
Image correcting means for correcting the captured display area image based on the relative positional relationship;
A portable terminal device comprising: an information reading unit that detects and analyzes an information display code included in the corrected display area image and reads information. The display area image includes an image indicating information on the information display code,
The mobile terminal device according to claim 1, wherein the information reading unit detects the information display code based on the information. A display area;
A marker provided for the mobile terminal device to identify the range of the display area from the entire image obtained by imaging the entire display device;
Image display means for displaying, in the display area, an image provided with an information display code representing predetermined information for the portable terminal device to read;
The display device characterized in that the marker includes display device specifying information provided for the mobile terminal device to specify the display device.

Images