JP2006153966A - Personal digital assistant - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a personal digital assistant in which superimposed display image information is recognized by a user. <P>SOLUTION: The system comprises; a holding section 3 including an information input section 4; a display section 1 having a display face 1a; a beam combiner 2 for guiding information displayed on the display face 1a and an image of a real visual field, to user's eyes E; a beam combiner opening/closing member 6; a display section opening/closing member 5; an imaging unit 7 arranged so that an incident light to the real visual field may be received; and an image processing unit 13. The imaging unit 7 images the real field and the image processing unit 13 has an area detecting process for detecting the area darker than a predetermined threshold value in the area of the image which is imaged by the imaging unit 7, and a display process for displaying information relating to a predetermined object in the area of the image which is imaged by the imaging unit 7 on the display area of the display face 1a corresponding to the dark area which is detected in the area detecting process in the display area of the display face 1a. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a portable information terminal device having a display element such as a liquid crystal panel.

  In recent years, a network society in which one person uses a plurality of computers, a so-called ubiquitous network society has been proposed. In the ubiquitous network society, it is possible to realize an environment where information networks such as the Internet can be accessed anytime and anywhere. The portable information terminal device used for such access is not limited to a personal computer, and for example, a mobile phone can be used.

For mobile phones, for example, Patent Document 1 proposes a portable information terminal device with a display function that can simultaneously perform a call and display recognition.
JP 2001-308997 A

  In the configuration disclosed in Patent Document 1, a small display is enlarged and projected far away by a hologram element. Thereby, for example, display image information can be seen while making a call. Further, a transmission image of a distant object can be simultaneously observed through the hologram element. This portable information terminal device with a display function can be used as a so-called see-through viewer that displays digital information superimposed on an image of a real field of view.

  In general, the reflectance of an object existing in a real field of view varies depending on each object. Therefore, when the see-through viewer is used, the brightness in the real field of view varies depending on the location in the real field of view. In the portable information terminal device with a display function of the prior art, depending on the position in the real field of view, the brightness of the object in the real field of view may be too bright relative to the brightness of the image information displayed on the display element. As a result, there is a problem that the user may not be able to recognize the superimposed display image information.

  The present invention has been made in view of the above, and an object of the present invention is to provide a portable information terminal device that allows a user to recognize superimposed display image information.

In order to achieve the above object, a portable information terminal device according to the present invention includes a gripping unit having an information input unit, a display unit having a display surface for displaying information input by the information input unit or information obtained from the outside, A beam combiner that reflects and displays information displayed on the display surface to the user's eye and transmits an image of an object in the real field in front of the user to the user's eye, and the beam combiner A beam combiner opening / closing member that supports the display unit so as to be openable / closable, a display unit opening / closing member that supports the display unit so that the display unit can be opened / closed, and light received from the real field side. An imaging unit having a plurality of light receiving surfaces arranged so as to perform an image processing unit, the imaging unit images the real field of view, and the image processing unit performs an area detection step and a display step. Yes The region detecting step compares a real field image captured by the imaging unit with a predetermined threshold, detects a dark region that is a region having a value lower than the predetermined threshold, and the display step In the display area of the display surface, related information related to a predetermined object in the real-field image area is displayed in the display area of the display surface corresponding to the dark area. It is characterized by that.
Moreover, it is preferable that the portable information terminal device according to the present invention varies the amount of the related information according to the area of the dark region.
In the portable information terminal device according to the present invention, it is preferable that the display color of the related information is changed according to the color of the dark area.
Moreover, it is preferable that the portable information terminal device according to the present invention displays an image for connecting the object and the dark area.

  ADVANTAGE OF THE INVENTION According to this invention, even if the brightness in a real visual field changes, the portable information terminal device which a user can recognize favorably the superimposed display image information can be provided.

Prior to the description of the embodiments, the effects of the present invention will be described.
FIG. 1 is an explanatory diagram of a portable information terminal device according to an embodiment. In FIG. 1, the position of the display part of a portable information terminal device and a beam combiner, and the relationship between these and the user's eye E are shown.

  The portable information terminal device includes a grip unit 3 having an information input unit 4, a display unit 1 having a display surface 1 a, and a beam combiner 2. The display surface 1a displays an image (information). The beam combiner 2 has an optical power for enlarging an image (information) displayed on the display surface 1a. The beam combiner 2 is composed of a half mirror. In addition, a display unit opening / closing member 5 is provided at a connection portion between the grip unit 3 and the display unit 1. Further, a beam combiner opening / closing member 6 is provided at a connection portion between the display unit 1 and the beam combiner 2. The display unit 1 is provided with a light receiving unit 7 including a light receiving element 7a.

  The image (information) displayed on the display surface 1 a of the display unit 1 is reflected by the beam combiner 2 and guided to the user's eye E. In addition, a virtual image of the image (information) displayed on the display surface 1a is enlarged and formed at a predetermined position separated from the user's eye E by the beam combiner 2. Therefore, the user can enlarge and observe this virtual image. Thus, the light passing through the beam combiner 2 enters the user's eye E and the light reflected by the beam combiner 2 enters.

  Here, an area that can be seen through the beam combiner 2 is defined as a real field of view. The brightness within this real field of view varies from place to place. For example, if there is a high building and a low building in the real field of view, depending on the position of the sun, the low building falls in the shadow of the high building, and the brightness of the low building is higher than that of the high building. Get dark.

  One surface 1c of the display unit 1 is a region that the user views through the beam combiner 2, that is, a surface that is substantially opposite to the real field of view. And the imaging unit 7 has the light receiving element 7a. The light receiving element 7a is composed of a plurality of light receiving surfaces, and is disposed at a position for receiving incident light from the front of the user. In the configuration of the portable information terminal device shown in FIG. 1, the user holds the portable information terminal device by holding the grip portion 3 with the right hand.

  Therefore, in the portable information terminal device according to the present embodiment, the imaging unit 7 is arranged on the surface 1c of the display unit 1 that is a surface substantially opposite to the real field of view to receive light from the real field of view.

  In such a state, the imaging unit 7 images the real field of view in front of the user. Then, the imaging unit 7 outputs a signal corresponding to the amount of light incident on each pixel. Subsequently, the output signal is input to the image processing unit 13. In the image processing unit 13, the input signal is compared with a predetermined threshold value. Through such a process, a dark area, which is an area where the brightness is lower than a predetermined threshold, is detected in the captured image.

  Here, the image picked up by the image pickup unit 7 is an image in the same region as the real field of view observed through the beam combiner 2. Therefore, the dark area detected by the above process corresponds to a dark area in the real field of view. The user sees the real field of view through the beam combiner 2. Therefore, a region corresponding to a dark region in the real field of view also exists on the optical surface of the beam combiner 2 as well.

  On the other hand, the optical surface of the beam combiner 2 reflects information displayed on the display surface 1a. Therefore, the predetermined position in the optical surface of the beam combiner 2 corresponds to the predetermined position in the display surface 1a.

The user is viewing the information displayed on the display surface 1 a through the beam combiner 2. Therefore, the dark area in the beam combiner 2 has a corresponding dark area in the display surface 1a.

  Therefore, the image processing unit 13 specifies the display area of the display surface 1a corresponding to the dark area of the real field of view. Then, the image processing unit 13 displays information related to a predetermined object in the area of the image captured by the imaging unit 7 in the specified display area of the display surface 1a. Hereinafter, “the object by the imaging unit 7...” Is simply referred to as “predetermined object”. As a result, information related to the predetermined object is displayed in the display area of the identified display surface 1a. This related information is superimposed on the real field image.

  By setting it as such a structure, the user can recognize the information relevant to a target object favorably in the display area of the specified display surface 1a.

  In the portable information terminal of the present embodiment, information related to a predetermined object is reflected on the specified display area of the display surface 1a by the beam combiner 2 and guided to the user's eye E. Yes.

  In the portable information terminal device having such a configuration, the beam combiner 2 is eccentrically arranged with respect to the display surface 1a. For this reason, decentration aberrations are likely to occur in an image observed through the beam combiner 2.

  However, as in this embodiment, the beam combiner 2 is given asymmetric power. For example, the optical surface of the beam combiner 2 is formed so that the power gradually changes in the direction along the virtual line L. With such a configuration, even if decentration aberration occurs due to the beam combiner 2 being decentered with respect to the display surface 1a, the decentration aberration can be corrected.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 2 is a schematic configuration diagram of the portable information terminal device according to the first embodiment as viewed from the side. Here, (a) shows a state where the display unit is closed, (b) shows a state where the display unit is expanded, and (c) shows a state where the beam combiner is expanded from the state shown in (b). FIG. 3 is an explanatory diagram showing the relationship between the display unit and the beam combiner in the portable information terminal device, and the position of the beam combiner and the user's eye E. FIG. 4 is an explanatory view showing the structure of the display unit opening / closing member 5 and the beam combiner opening / closing member 6.

  The portable information terminal device according to the first embodiment includes a grip unit 3 having an information input unit 4, a display unit 1 having a display surface 1a, a beam combiner 2 having optical power, and a display unit opening / closing member 5. Yes. Here, the display surface 1a displays information input by the information input unit 4 or information received from the outside. The display unit opening / closing member 5 supports the display unit 1 so as to be openable and closable with respect to the grip unit 3. With such a configuration, the portable information terminal device according to the first embodiment is configured as a so-called folding type mobile phone.

  Further, the beam combiner 2 is connected to the display unit 1 via the beam combiner opening / closing member 6. This connection position is one end side of the display unit 1 and is a position facing the display unit opening / closing member 5. With this beam combiner opening / closing member 6, the beam combiner 2 can be fixed to the display unit 1 at a predetermined angle. With such a configuration, the portable information terminal device of Example 1 is also configured as a see-through viewer as shown in FIG.

  As shown in FIG. 4, the display portion opening / closing member 5 includes a cylindrical portion 5a, a cylindrical portion 5b, a spring 5c, a spherical member 5d, a pin 5e, and a stopper 5f. Here, the cylindrical portion 5 b is integrally formed at one end of the display portion 1. The cylindrical portion 5 a is integrally formed at one end of the grip portion 3.

A through hole 5a ₁ is formed in the center of the cylindrical portion 5a along the longitudinal direction. A concave portion 5a ₂ for accommodating the head of the pin 5e is formed at one end of the cylindrical portion 5a, and an attachment portion (not shown) for attaching the spring 5c is formed at the other end.

A through hole 5b ₁ is formed at the center of the cylindrical portion 5b along the longitudinal direction. A recess (not shown) for accommodating the stopper 5f is provided at one end of the cylindrical portion 5b, and locking grooves 5b ₂ and 5b ₃ into which a part of the spherical member 5d can be fitted at the other end. Are formed.

And in the display part opening-and-closing member 5, while attaching the spring 5c to the attachment part of the cylindrical part 5a, the spherical member 5d is attached to the front-end | tip of the spring 5c. In this state, the pin 5e from the recess 5a ₂ side of the cylindrical portion 5a, through penetration of the cylindrical portion 5a and 5b hole 5a ₁ and 5b _1, has stopped its tip stopper 5f. With such a configuration, the display unit opening / closing member 5 supports the display unit 1 so as to be openable and closable with respect to the grip unit 3.

In the display portion opening / closing member 5, when the display portion 1 is deployed, the spherical member 5d is fitted into the locking grooves 5b ₂ and 5b ₃ and is urged by the spring 5c. By setting it as such a structure, the display part 1 and the holding part 3 can be fixed now at arbitrary angles.

  The specific configuration of the display unit opening / closing member 5 is not limited to the above configuration. For example, each of the cylindrical portions 5a and 5b may be provided with a locking portion provided with a magnet, and the display portion 1 and the grip portion 3 may be fixed when the locking portions contact each other. .

Beam combiner closing member 6 has a cylindrical portion 6a, a locking portion 6b, and the cylindrical portion 6c, and the locking claw 6c _2, has a pin and stop (not shown). Here, the cylindrical portion 6 a and the locking portion 6 b are integrally formed on the other end of the display portion 1. On the other hand, the cylindrical portion 6 c is integrally formed at one end of the beam combiner 2. Further, the locking claw 6c ₂ is integrally formed with the cylindrical portion 6c. The cylindrical portions 6a and 6c are formed with through holes 6a ₁ and 6c _{1 at} the center, respectively. Magnets respectively provided on the locking portion 6b and the locking claws 6c _2.

The beam combiner opening / closing member 6 is configured such that a pin (not shown) is passed through the through holes 6a ₁ and 6c ₁ of the cylindrical portions 6a and 6c, and the tip thereof is stopped by a stopper (not shown). With such a configuration, the beam combiner opening / closing member 6 supports the beam combiner 2 so as to be openable / closable with respect to the display unit 1.

The beam combiner closing member 6, when expanding a beam combiner 2, locking claws 6c ₂ is brought into contact with the locking portion 6b, and, by holding the state in a mutual magnetic force, a beam combiner 2 It can be fixed to the display unit 1.

The beam combiner opening / closing member 6 is not limited to the above configuration. For example, the spherical member 5d in the display unit opening / closing member 5 may be configured in the same manner as the configuration in which the spherical member 5d is fitted in the locking grooves 5b ₂ and 5b ₃ and is urged by the spring 5c.

  With such a configuration, as shown in FIG. 3, the information displayed on the display surface 1a is reflected in the direction of the user's eye E. As described above, the portable information terminal device according to the first embodiment can be used as a mobile phone and as a see-through viewer.

  In the portable information terminal device of the first embodiment, the imaging unit 8 is provided in the display unit 1 as a configuration corresponding to the light receiving unit 7 in FIG. The imaging unit 8 images the outside via the surface 1c. The surface 1c is a surface that is substantially opposite to the real field of view.

  5A and 5B are explanatory views showing the structure of the imaging unit provided in the information display apparatus of Example 1, wherein FIG. 5A is a schematic configuration diagram of the imaging unit, and FIG. The relationship between the viewing direction and the optical axis direction of the imaging lens is shown. As shown in FIG. 5A, the imaging unit 8 includes an imaging lens 8a, an imaging element 8b, and a cover glass 8c. The imaging lens 8a includes a prism, a negative lens, and a positive lens. Note that the cover glass 8c serves as an imaging window. The image sensor 8b is a light receiving element. In the first embodiment, the image sensor 8b is a CCD, and includes a plurality of light receiving units.

  In the first embodiment, when the display unit 1 is unfolded, the display unit 1 is bent with respect to the grip unit 3 at an angle of 180 ° or more. Therefore, as shown in FIG. 5B, the direction of the arrow X1 and the direction of the arrow X2 are different. Here, the direction of the arrow X1 is a direction in which the user views the real field of view through the beam combiner 2. On the other hand, the direction of the arrow X2 is the direction of the normal line on the surface 1c.

  Therefore, if the imaging lens 8a is arranged so that the optical axis of the imaging lens 8a is parallel to the direction of the arrow X2, it is impossible to capture the real field of view that the user is viewing. Therefore, by arranging the imaging lens 8a so that the optical axis of the imaging lens 8a is parallel to the direction of the arrow X3, the imaging element 8b images in the real field of view (predetermined imaging region).

  The portable information terminal device has a built-in memory 12. Therefore, an image captured by the image sensor 8 b provided in the imaging unit 8 is temporarily stored in the memory 12. Further, the portable information terminal device has an image processing unit 13 built therein. Then, predetermined processing is performed in the image processing unit 13.

  FIG. 6 is a flowchart illustrating a process of processing an image captured by the imaging unit 8 in the image processing unit 13. First, the image processing unit 13 reads an image captured by the imaging unit 8 from the memory 12 (step S1). Here, the image is composed of a plurality of pixels. Therefore, the image processing unit 13 compares the brightness of each pixel with a threshold value (step S2).

  Here, when the brightness of the pixel is smaller than the threshold, the image processing unit 13 stores the coordinates of the pixel (step S3). On the other hand, when the pixel brightness is equal to or higher than the threshold, the image processing unit 13 does nothing. The image processing unit 13 performs the above process for all the pixels (step S4).

  When the comparison is completed for all the pixels, the image processing unit 13 specifies a collection of images with continuous coordinates, that is, a dark region, from the stored coordinates (step S5). Here, the number of dark regions varies depending on the state of the imaging region. Therefore, the processing is varied according to the number of dark areas (step S6).

  When there are a plurality of dark regions, the dark region with the largest area corresponds to a dark region for display for displaying information related to a predetermined object (step S7). Hereinafter, the “predetermined object... Dark region” is simply referred to as a “dark region for display”.

  On the other hand, when there is only one dark area, this dark area corresponds to a dark area for display.

  When the dark area for display is specified, the image processing unit 13 specifies the dark area for display in the actual visual field. Further, based on this specification, the display area of the display surface 1a corresponding to the dark area for display is specified. (Step S8).

  When the display area of the display surface 1a is specified, the image processing unit 13 displays information on the specified display area of the display surface 1a (step S9).

  Here, the displayed information is information related to a predetermined object. The related information related to the predetermined object may be configured to be retrieved and extracted from a database provided separately from the portable information terminal device or built in the portable information terminal device.

  A user may select an object for which information is to be displayed in advance. As this selection method, for example, there is a method of providing an eye gaze detection device and specifying an object in the imaging region. Further, there is a method in which an arrow is displayed on the display surface 1 a and the arrow can be operated by the user using the information input unit 4.

  Thus, the information displayed on the display surface 1a is reflected by the beam combiner 2 toward the user's eye E. At this time, as shown in FIG. 7, the information is displayed superimposed on a dark region in the real field of view. Therefore, the user can satisfactorily see both the object in the real field of view and the information displayed on the display surface 1a.

  In the configuration shown in FIG. 5, an imaging window (cover glass 8c) is provided on the side surface 1c. However, it is not necessarily limited to such a configuration. FIG. 8 is an explanatory view showing a modification of the structure of the imaging unit 8 provided in the information display device.

  For example, as shown in FIG. 8A, if a prism constituting the imaging lens 8a is projected from the display unit 1, an imaging window unit is provided on another side surface 1c ′ of the display unit 1. Can be provided.

  Further, as shown in FIG. 8B, the imaging unit 8 may be arranged in a direction orthogonal to the arrangement of the imaging unit 8 in FIG. In either case, the image pickup unit 8 is arranged so that the optical axis of the image pickup lens a is directed in the direction of the real field of view.

  As described above, when the portable information terminal device according to the first embodiment is used as a see-through viewer, the portable information terminal device is configured to hold the portable information terminal device in contact with the temporal region. The imaging unit 8 is arranged so that the optical axis of the lens 8a is oriented in the real visual field direction.

  According to the portable information terminal device of the first embodiment, the imaging unit 8 captures a real field of view viewed by the user, and the image processing unit 13 detects a change in brightness within the real field of view from the captured image. Then, information can be displayed by specifying the display area of the display surface 1a corresponding to the dark area of the real field of view. Therefore, the information to be displayed on the display surface 1a can be displayed in a dark area in the real field so as to be superimposed on the image of the real field. As a result, the user can recognize the display image information obtained by superimposing information on the real field of view and the captured image.

  FIG. 9 is a perspective view of the schematic configuration of the portable information terminal device according to the second embodiment as viewed from the side. Here, (a) shows a state where the display unit is closed, (b) shows a state where the display unit is expanded, and (c) shows a state where the beam combiner and the display surface configuration unit are expanded from the state (b). . FIG. 10 is an explanatory diagram showing the relationship between the display unit and the beam combiner in the portable information terminal device, and the position of the beam combiner and the eye E of the user.

  The portable information terminal device according to the second embodiment includes a grip unit 3 having an information input unit 4, a display unit 1, a beam combiner 2 having optical power, and a display unit opening / closing member 5. Here, the display unit 1 includes a display surface 1a and an outer frame portion 1b surrounding the display surface 1a. In the figure, reference numeral 1a 'denotes a display surface constituting part including the display surface 1a. Here, the display surface 1a displays information input by the information input unit 4 or information received from the outside. Moreover, the display part opening / closing member 5 supports the outer frame part 1 b of the display part 1 so as to be openable and closable with respect to the grip part 3. With such a configuration, the portable information terminal device according to the first embodiment is configured as a so-called folding type mobile phone.

  Further, the beam combiner 2 is connected to the display unit 1 via the beam combiner opening / closing member 6. This connection position is one end side of the display unit 1 and is a position facing the display unit opening / closing member 5. With this beam combiner opening / closing member 6, the beam combiner 2 can be fixed at a predetermined angle with respect to the display unit 1. With this configuration, the portable information terminal device according to the second embodiment is configured as a see-through viewer as shown in FIG.

  The display part opening / closing member 5 and the beam combiner opening / closing member 6 are as described in the first embodiment. The display surface rotating means 9 has a configuration that supports the display surface 1a so as to be rotatable with respect to the outer frame portion 1b, and holds the state when the display surface 1a is rotated by a predetermined amount. As the display surface rotating means 9, a known rotating mechanism may be used. Therefore, detailed description of a specific configuration is omitted.

  And in the portable information terminal device of Example 2, the imaging unit 8 is provided in the outer frame part 1b as a structure corresponded to the light reception unit 7 of FIG. The imaging unit 8 has a plurality of light receiving surfaces as in the first embodiment. Also in the second embodiment, the imaging unit 8 captures an image within the actual visual field (predetermined imaging region). Then, the dark area is detected in the same manner as in the first embodiment.

  Here, the area of the dark region varies depending on the state in the real field of view. Thus, in the second embodiment, the amount of information to be displayed is varied according to the area of the dark region. FIG. 11 is an explanatory diagram showing a state in which information is superimposed and displayed in a dark region of the real field of view. As shown in FIG. 11A, when the area of the dark region is large, the amount of information to be displayed is increased. On the other hand, as shown in FIG. 11B, when the area of the dark region is small, the amount of information to be displayed is reduced.

  According to the portable information terminal device of the second embodiment, the amount of information to be displayed on the display surface 1a can be varied according to the area of the dark region in the real field of view (predetermined imaging region). The user can effectively recognize the display image information in which the information based on the captured image is efficiently superimposed on the real field of view.

  Note that the amount of information to be displayed is increased in proportion to the area of the dark region. Therefore, when the area of the dark region is small as in FIG. 11B, the amount of information to be displayed is reduced. Therefore, all information may not be displayed at once.

  Therefore, the information displayed in the dark area may be displayed as the following information when a predetermined time has elapsed. Alternatively, the following information may be displayed by operating the information input unit 4.

  FIG. 12 is a perspective view of the schematic configuration of the portable information terminal device according to the third embodiment as viewed from the side. Here, (a) shows a state where the display unit is closed, (b) shows a state where the display unit is expanded, and (c) shows a state where the beam combiner is expanded from the state (b). FIG. 13 is an explanatory diagram showing the relationship between the display unit and the beam combiner in the portable information terminal device, and the position of the beam combiner and the eye E of the user.

  The portable information terminal device according to the third embodiment includes a grip unit 3 having an information input unit 4, a display unit 1 having a display surface 1a, a beam combiner 2 having optical power, and a display unit opening / closing means 5. ing. Here, the display surface 1a displays information input by the information input unit 4 or information received from the outside. The display unit opening / closing means 5 supports the display unit 1 so that it can be opened and closed with respect to the grip unit 3. With such a configuration, the portable information terminal device according to the first embodiment is configured as a so-called folding type mobile phone.

  The beam combiner 2 is connected to the display unit 1 via a beam combiner opening / closing member 6, a beam combiner connecting member 10, and a beam combiner moving member 11. With such a configuration, the portable information terminal device of Example 3 is also configured as a see-through viewer as shown in FIG.

  The display portion opening / closing member 5 is as described in the first embodiment. The beam combiner connecting member 10 is a rod-shaped member. The beam combiner opening / closing member 6 and the beam combiner moving member 11 may use a known rotation mechanism. Therefore, detailed description of a specific configuration is omitted.

  The beam combiner connecting member 10 is connected to the beam combiner 2 at one end and to the display unit 1 at the other end. The beam combiner moving member 11 supports the beam combiner connecting member 10 so as to be movable with respect to the display unit 1. The moving direction is a direction in which the beam combiner 2 moves away from the display surface 1a.

  The beam combiner moving member 11 has a mechanism that can maintain the state at that position when the beam combiner connecting member 10 is moved by a predetermined amount. The beam combiner opening / closing member 6 supports the beam combiner 2 so as to be openable / closable with respect to the beam combiner connecting member 10. The beam combiner opening / closing member 6 has a mechanism that can maintain the state at the position when the beam combiner 2 is deployed a predetermined amount.

  And in the portable information terminal device of Example 3, the imaging unit 8 is provided in the display part 1 as a structure corresponded to the light reception unit 7 of FIG. The imaging unit 8 has a plurality of light receiving surfaces as in the first embodiment. Also in the third embodiment, the imaging unit 8 captures an image within the actual visual field (predetermined imaging area). Then, the dark area is detected in the same manner as in the first embodiment.

  In the third embodiment, since the imaging unit 8 is a color CCD, color information of a dark area can be obtained. Therefore, in the third embodiment, the color that is most easily recognized is used as the information display color according to the color of the dark region.

  FIG. 14 is an explanatory diagram showing a state in which information is superimposed and displayed on a dark area of the real field of view, and shows the relationship between the color of the dark area and the color of the displayed information. When the color of the dark region 1 is the color A, the color for displaying information is the color α. When the color of the dark region 2 is the color B, the color for displaying information is the color β. When the color of the dark region 3 is the color C, the color for displaying information is the color γ. The color A and the color α are different from each other. The same applies to the color B and the color β, and the color C and the color γ.

  In addition, when the dark region and the target object are separated from each other, information on the target object is displayed at a position away from the target object. In this case, information relating to the predetermined object and an image connecting the predetermined object may be displayed. For example, as shown in FIG. 15, an arrow may be displayed in order to link information displayed in a dark area with an object.

  The portable information terminal device according to the third embodiment can also obtain the same effects as those of the first and second embodiments.

It is explanatory drawing in the portable information terminal device concerning one Embodiment of this invention. It is the schematic block diagram which looked at the portable information terminal device of Example 1 from the side, (a) is the state which closed the display part, (b) is the state which expand | deployed the display part, (c) is (b). In this state, the beam combiner is deployed. It is explanatory drawing which shows the relationship between the display part in the portable information terminal device of Example 1, and a beam combiner, and the position of a beam combiner and a user's eyes. It is explanatory drawing which shows the structure of a display part opening / closing member and a beam combiner opening / closing member. BRIEF DESCRIPTION OF THE DRAWINGS It is explanatory drawing which shows the structure of the imaging unit with which the information display apparatus of Example 1 was equipped, Comprising: (a) is a schematic block diagram of an imaging unit, (b) is a direction where a user views an actual visual field through a beam combiner, and It is a figure which shows the relationship with the optical axis direction of an imaging lens. It is a flowchart which shows the process which processes the information imaged with the imaging unit in an image processing unit. It is a figure which shows a mode that information is superimposed and displayed on the dark area | region of a real visual field. It is explanatory drawing which shows the modification of the structure of the imaging unit with which the information display apparatus was equipped, Comprising: (a) is the case where the window part for imaging is provided in another side surface of a display part, (b) is FIG. This is a case where it is arranged in a direction orthogonal to the imaging unit arrangement position. It is the perspective view which looked at schematic structure of the portable information terminal device concerning Example 2 from the side, (a) is the state which closed the display part, (b) is the state which expand | deployed the display part, (c) is ( In this state, the beam combiner is expanded from the state of b). It is explanatory drawing which shows the relationship between the display part in the portable information terminal device of Example 2, and a beam combiner, and the position of a beam combiner and a user's eyes. It is explanatory drawing which shows a mode that information is superimposed and displayed on the dark area | region of a real visual field, Comprising: (a) The amount of information displayed when the area of a dark area is large, (b) The area of a dark area It is a figure which shows the information content displayed when it is small. It is the perspective view which looked at schematic structure of the portable information terminal device concerning Example 3 from the side, (a) is the state which closed the display part, (b) is the state which expand | deployed the display part, (c) is ( In this state, the beam combiner is expanded from the state of b). It is explanatory drawing which shows the relationship between the position of the display part in the portable information terminal device of Example 3, a beam combiner, and a beam combiner and a user's eyes. It is explanatory drawing which shows a mode that information is superimposed and displayed on the dark area | region of a real visual field, Comprising: The relationship between the color of a dark area | region and the color of the information displayed is shown. It is explanatory drawing which shows a mode that information is superimposed and displayed on the dark area | region of a real visual field, Comprising: When a dark area | region and the target object have left | separated, it shows a mode that the image which associates both is displayed.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Display part 1a Display surface 1a 'Display surface structure part 1b Outer frame part 1c (1c') One surface (side surface) of display part
2 Beam combiner 3 Gripping part 3a, 3b One side (side) of the gripping part
4 information input unit 5 display unit closing means 5a, 5b cylindrical portion 5a ₁ 1 and 5 b ₁ through hole 5a ₂ recesses 5b _2, 5b ₃ engaging grooves 5c spring 5d spherical member 5e pins 5f stop 6 beam combiner switching means 6a, 6c Cylindrical parts 6a ₁ , 6c ₁ Through-hole 6b Locking part 6c ₂ Locking claw 7 Light receiving unit 8 Imaging unit 8a Imaging lens 8b Imaging element 8c Cover glass 9 Display surface rotating means 10 Beam combiner connecting member 11 Beam combiner moving member 12 Memory 13 Image processing unit E User's eye L Virtual line

Claims (4)

A gripping part having an information input part;
A display unit having a display surface for displaying information input in the information input unit or information obtained from the outside;
A beam combiner that reflects and displays information displayed on the display surface to the user's eye and transmits an image of an object in the real field in front of the user to the user's eye;
A beam combiner opening and closing member that supports the beam combiner so as to be opened and closed with respect to the display unit;
A display unit opening and closing member that supports the display unit so as to be openable and closable with respect to the gripping unit;
An imaging unit having a plurality of light receiving surfaces arranged to receive light incident from the real field side;
An image processing unit,
The imaging unit images the real field of view,
The image processing unit has a region detection step and a display step,
The region detection step compares the image of the real field of view captured by the imaging unit with a predetermined threshold, detects a dark region that is a region having a value lower than the predetermined threshold,
The display step displays related information related to a predetermined object in the real field image area in the display area of the display surface corresponding to the dark area of the display area of the display surface. A portable information terminal device characterized by being configured as described above.   The portable information terminal device according to claim 1, wherein the amount of the related information varies according to the area of the dark region.   The portable information terminal device according to claim 1 or 2, wherein a display color of the related information is made different according to a color of the dark region.   The portable information terminal device according to any one of claims 1 to 3, wherein an image for connecting the object and the dark region is displayed.