JP2014146871A - Wearable display device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a wearable display device and a program, etc. for properly displaying content data by controlling a frame area on the circumference of a display image, on the basis of the content data.SOLUTION: The wearable display device comprises: a processor 100 which, on the basis of the content data to be displayed, controls at least one of the shape and luminance of a frame area set in the circumference of a display image, and generates the display image containing the frame area and a content display area for displaying the content data; a display controller 200 which performs display control of the generated display image; and a display 300 which, on the basis of the display control by the display controller 200, performs see-through display of the generated display image superimposed on the environment.

Description

  The present invention relates to a wearable display device, a program, and the like.

  In recent years, when presenting image information to a user, a wearable display device (for example, an HMD mounted on the head, a head mounted display) that is worn on the body of the user has been used. Although several types of wearable display devices are possible, in a see-through type wearable display device, a display image is superimposed and displayed on a part of the external field of view.

  For example, Patent Document 1 describes an HMD as a specific example of a stereoscopic video display, and a configuration of a binocular stereoscopic HDM that uses a technique in which a contour portion of a display area is blurred and displayed naturally to enhance a sense of reality. Is disclosed. In Patent Document 1, the width and position of the blur frame are determined by adjusting the content position so that the parallax is substantially constant. Patent Document 1 relates to a stereoscopic image display method, and is not limited to a see-through wearable display device.

  Further, Patent Document 2 discloses a configuration of a see-through type HMD for the purpose of simultaneously grasping content data such as a manual and the outside world. In Patent Document 2, the layout of the content is dynamically changed so that the display content and the imaging range of the camera do not overlap, and then the brightness of the area overlapping the imaging range of the camera is lowered to display the external image.

Japanese Patent Laid-Open No. 9-322199 JP 2011-76503 A

  In a see-through wearable display device, the display image is superimposed and displayed on the external field of view, which may cause a sense of incongruity due to the conspicuous boundary between the external field of view and the display image (the contour of the display image). There is. In addition, since the display image is displayed in a smaller area than the entire real field of view of the user, a small display unit (for example, an eyepiece window) may be used, and the influence of vignetting cannot be ignored due to the relationship with the size of the pupil. There is also sex.

  Patent Documents 1 and 2 disclose a technique for displaying a display image naturally or appropriately presenting the contents of content to a user in a wearable display device. However, the degree of discomfort described above and the degree of influence of vignetting depend on the content of the original content to be displayed. However, in Patent Document 1, the content of the content is not considered in setting the blur frame. Further, according to the method of Patent Document 2, it is possible to easily grasp the outside world while viewing the content, but the point that the outline of the display area is clearly recognized is not particularly taken into consideration.

  According to some aspects of the present invention, it is possible to provide a wearable display device, a program, and the like that appropriately display content data by controlling a frame region at the periphery of the display image based on the content data. Can do.

  According to one embodiment of the present invention, content for displaying the frame region and the content data by controlling at least one of the shape and luminance of the frame region set at the periphery of the display image based on content data to be displayed A processing unit that generates the display image including a display area, a display control unit that performs display control of the generated display image, and a display unit that is generated based on the display control of the display control unit. The present invention relates to a wearable display device including a display unit that displays the display image on the outside world in a see-through manner.

  In one embodiment of the present invention, a frame region is controlled based on content data, and a display image including the frame region is displayed. As a result, an appropriate frame area can be set according to the content data, so that the content can be effectively displayed.

  In one aspect of the present invention, the content data includes first content data and second content data different from the first content data, and the processing unit corresponds to the first content data. A region in which at least one of the shape and the luminance is different between the first frame region of the first display image and the second frame region of the second display image corresponding to the second content data is set. May be.

  Thereby, it is possible to set an appropriate frame area according to the content data.

  In one aspect of the present invention, the processing unit performs control of the shape such that the area of the frame region is zero with respect to at least one of the first display image and the second display image, and the frame region As the display image in which the area is zero, the display image in which the content display region is entirely displayed without including the frame region may be generated.

  As a result, it is possible to control the area of the frame area to be 0 as the control of the shape of the frame area.

  In one aspect of the present invention, when the second content image information included in the second content data is an image obtained by enlarging a part of the first content image information included in the first content data, Alternatively, when the angle of view of the second content image information is narrower than the angle of view of the first content image information, the processing unit includes the first frame included in the first display image. The area of the second frame area included in the second display image may be set larger than the area of the area.

  As a result, the area of the frame region can be controlled according to the degree of expansion of the content data or the angle of view.

  In one aspect of the present invention, the processing unit performs recognition processing for recognizing a human face area from the first content image information and the second content image information, and based on the recognition processing, It may be determined whether or not the content image information is an image obtained by enlarging a part of the first content image information.

  As a result, the degree of expansion of the content data can be performed by face area recognition processing.

  In one aspect of the present invention, the processing unit may control at least one of the shape and the luminance of the frame region based on the luminance of at least a part of the display image including the content display region.

  This makes it possible to control the frame area based on the luminance.

  In one aspect of the present invention, the processing unit may control the frame region to be a square or a circle as control of the shape of the frame region.

  This makes it possible to perform control so that the shape of the frame region is square or circular.

  In one aspect of the present invention, the processing unit may perform control of the frame region in a gradation display in which the luminance of the frame region monotonously decreases in a direction from the central portion to the peripheral portion of the display image. .

  Thereby, by setting the frame area to gradation display, it becomes possible to enhance the setting effect of the frame area compared to the case where the frame area is not set or the frame area is a single color.

  In one aspect of the present invention, the processing unit may generate the display image by performing a blurring process at a boundary between the frame region and the content display region.

  As a result, by performing the blurring process, it is possible to enhance the setting effect of the frame area compared to the case where the frame area is not set or the frame area is a single color.

  In one aspect of the present invention, the processing unit may perform at least one of a size change process and a trimming process on content image information included in the content data based on an inner diameter of the frame region. .

  Accordingly, it is possible to generate a display image after performing a size changing process on the image represented by the content image information based on the size of the frame region.

  In one aspect of the present invention, the processing unit determines a region of interest in the content image information included in the content data, and adds the frame region to the determined region of interest so that the display is performed. An image may be generated.

  This makes it possible to use the attention area in the content image information as a display image.

  In one aspect of the present invention, the processing unit determines the attention area in the content image information based on metadata included in the content data, and the frame area with respect to the determined attention area The display image may be generated by adding.

  This makes it possible to determine a region of interest based on metadata.

  In one aspect of the present invention, the processing unit is a result of a motion detection process that detects a movement of a subject on the content image information, or a subject tracking process that detects a position of the target subject on the content image information. The display image may be generated by determining the attention area in the content image information and adding the frame area to the determined attention area.

  As a result, it is possible to determine a region of interest by motion detection processing or subject tracking processing.

  In one aspect of the present invention, the content data includes urgency information indicating an urgency level, and the processing unit controls at least one of the shape and the luminance of the frame region based on the urgency level information. May be.

  This makes it possible to control the frame area using the urgency information.

  In one aspect of the present invention, the processing unit may control a display mode of the content data based on the urgency information.

  Thereby, in addition to the control of the frame area, it is possible to perform display control of content data using the urgency information.

  In one aspect of the present invention, an eyepiece window that emits a light beam that forms the display image to be displayed on the display unit toward a user is provided, and the eyepiece window has a projected cross-section in the visual axis direction of the user. The width may be 4 mm or less.

  As a result, it becomes possible to realize a wearable display device of the pupil division see-through method as a see-through wearable display device.

  In one aspect of the present invention, the display control unit may control the luminance of the frame region by controlling a display mode of the display image.

  As a result, the brightness control of the frame area can be performed in the display control unit.

  In one aspect of the present invention, the display control unit may control the luminance of the frame region by controlling a backlight of the display unit.

  As a result, the brightness control of the frame area can be performed by the backlight control in the display control unit.

  Another aspect of the present invention is a content display that displays the frame region and the content data by controlling at least one of the shape and luminance of the frame region at the periphery of the display image based on the content data to be displayed. The present invention relates to a program that causes a computer to function as a processing unit that generates a display image including a region and a display control unit that performs control to display the generated display image on the display unit in a superimposed manner with the outside world.

The example of mounting | wearing with the wearable display apparatus of this embodiment. 2A and 2B are explanatory diagrams of a user's real field of view and display area. 3A and 3B are explanatory diagrams of the relative sizes of the pupil and the eyepiece window holding unit. 4 (A) and 4 (B) are explanatory diagrams of a decrease in the visibility of the display image. FIG. 5A and FIG. 5B are explanatory diagrams of changes in pupil size that cause vignetting. An example of a display area when vignetting occurs. 2 shows a configuration example of a wearable display device. 8A to 8C show examples of display images when a rectangular frame region is set. An example of a display image when a circular frame region is set. The flowchart explaining the process of 1st Embodiment. FIG. 11A and FIG. 11B are display examples of emergency bulletins and the like. The flowchart explaining the process of 2nd Embodiment. The example which controls a frame area | region based on metadata. FIGS. 14A to 14C show examples in which a part of a content image is used as a display image. FIGS. 15A to 15C show examples of determining the screen center based on the motion detection process. FIGS. 16A and 16B show examples in which the image center is determined based on subject tracking processing. The flowchart explaining the process of 3rd Embodiment. 18A and 18B show examples in which a frame region is set according to the average luminance. FIG. 19A and FIG. 19B are other examples in which a frame region is set according to the average luminance. FIGS. 20A and 20B are examples in which the scaling process of the content image is performed according to the inner diameter of the frame region. The flowchart explaining the process of 4th Embodiment.

  Hereinafter, this embodiment will be described. In addition, this embodiment demonstrated below does not unduly limit the content of this invention described in the claim. In addition, all the configurations described in the present embodiment are not necessarily essential configuration requirements of the present invention.

1. First, the method of this embodiment will be described. In recent years, wearable information display devices have attracted attention as information terminals for accessing information whenever a user desires anytime and anywhere. For example, a head-mounted information display device (HMD) can be confirmed by simply wearing the head-mounted information display device (HMD). You don't have to take it out of your bag or pocket every time you want information like a smartphone. In order to always wear it, it is necessary to make it smaller and lighter and to have a see-through configuration that can always observe the visual information of the outside world. Some wearable display devices do not observe the visual information of the outside world, but this is used in situations where immersive feelings are important, such as watching movies. Is not supposed to be installed.

  Examples of the see-through HMD include a half mirror method, a HOE (holographic optical element) method, a retinal scanning method, and a pupil division see-through method. Hereinafter, a pupil division see-through HMD that is easy to downsize and has excellent see-through performance without hindering the external field of view by display contents will be described. However, the wearable display device of the present embodiment is not limited to this, and other methods may be used.

  An example of wearing a wearable display device is shown in FIG. In the example shown in 11 of FIG. 1, the device is mounted with the ear portion as a fulcrum. For example, this may be realized by a shape to be put on the ear like glasses. Then, the display unit is used so as to be positioned in front of the user's eyeball by adjusting the holding unit that holds the display unit.

  FIGS. 2A and 2B show examples of the user field of view when a see-through type wearable display device is used. FIG. 2A shows a visual field when a wearable display device is not attached, that is, a real visual field, and a house, a mountain, or the like that is actually present in the user's line of sight is observed. It will be. FIG. 2B shows an example of the user field of view when the wearable display device is attached. In the example of FIG. 2B, the display image presented to the user by the display unit is observed in the lower right portion of the entire user field of view as shown in DA. Here, a display image including text is displayed, and as can be seen from the comparison between FIG. 2A and FIG. 2B, even when the display image is superimposed and displayed, many parts of the external field of view are displayed. Observable.

  3A and 3B show the relationship between the display unit (eyepiece window) and the pupil in the pupil division see-through method. The size (pupil diameter) of the human pupil PU shown in FIG. 3A changes between 2 and 8 mm depending on the brightness. When the shielding object arranged in front of the eyeball is smaller than the pupil diameter, the distant scenery can be observed without being obstructed by the shielding object. That is, by making the eyepiece window holding unit HO for displaying content smaller than the pupil PU as shown in FIG. 3B, the light flux of the electronic image emitted from the eyepiece window without blocking the light flux from the outside. Can be confirmed, and the image of the outside world (real field of view) and the electronic image can be visually recognized by overlapping (overlapping). Thereby, a see-through effect is obtained. This is the principle of the pupil division see-through method. Here, the width of the eyepiece window holding part is desirably 4 mm or less as shown in FIG. 3B with reference to the average pupil diameter size.

  In such a see-through type wearable display device, several problems may occur with respect to the visibility of the display image. The first problem is a decrease in visibility due to the boundary between the external field of view and the display image. Specifically, the content clearly recognizes the boundary between the display area and the non-display area, so that the degree of attention to the content may be relatively lowered. As a result, the naturalness of content display secured by the fusion of the outside world and content through see-through is impaired.

  For example, as shown in FIG. 4A, even in a see-through type head mounted display, there is a case where an eyepiece window holding part (HO in FIG. 4A) such as a mirror frame or a half mirror is conscious. . When the brightness of the display content is high and the display is reflected on the mirror frame, the boundary BO between the non-display area NA and the display area DA becomes clear, and the unnaturalness of content display becomes remarkable. Further, as shown in FIG. 4B, even when the holding portion of the eyepiece window is not conscious, the luminance difference of the boundary BO is unnaturally felt when the luminance is greatly different from the external field of view of the display image. There is a case.

  As a second problem, vignetting can be considered. As shown in FIGS. 5A and 5B, the relative size between the pupil and the eyepiece window holding unit HO changes as the size of the pupil PU changes due to a change in the amount of light in the outside world. When the amount of external light is sufficient and the pupil is small with respect to the eyepiece window holding portion as shown in FIG. 5B, there is a high possibility that the screen will be vignetted and the periphery of the screen will not enter the field of view. FIG. 6 shows an example of the user field of view in such a situation. As can be seen from the comparison with FIG. 2B, the peripheral portion of the display image cannot be visually recognized by vignetting, so that the content is displayed in the central portion A1 and not in the peripheral portion A2 in the display area DA. ing. In particular, when the content is at the periphery of the screen, the image is interrupted and it is not preferable because it causes stress on the user. For example, when a part of the text is also arranged at the periphery as in FIG. 6, the text is missing and information cannot be browsed sufficiently, and the user's stress is great.

  Therefore, the present applicant provides a frame region in the peripheral portion of the display image so that the boundary between the display region and the non-display region (external field visual field) in the user visual field is not conspicuous, and the display image (in a narrow sense). Proposes a method for improving the visibility of the content presented there. In this way, it is possible to suppress a decrease in visibility due to the boundary, and since content is not displayed in the first place in an area where vignetting occurs, occurrence of user stress due to lack of content can be suppressed. At this time, the difference from the technique disclosed in Patent Document 1 is that the shape and brightness of the frame area are determined based on the content data.

  In general, the higher the luminance of the display image, the clearer the boundary between the display area and the non-display area in the user's field of view, and there is a tendency that the uncomfortable feeling at the time of display increases. Therefore, when the luminance of the original content is high (for example, an image captured in the daytime or when black characters are displayed on a white background), and when the luminance of the content is low (for example, an image captured of a night view or white on a black background). For example, when displaying characters, a different frame area should be set. However, the conventional method has not been processed from this point of view. In the conventional method, since the frame area is set regardless of the content, there is a problem that the effect of reducing the sense of discomfort is not sufficient for content with high brightness, or an excessive frame area is set for content with low brightness. In other words, there may be a problem that the visibility of the content is impaired (for example, the content is crushed by the frame area or the content needs to be reduced). Also, the appropriate frame region may change depending on other factors caused by the content such as the position of the subject of interest in the image representing the content (content image), but this is not considered in the conventional method.

  In the present embodiment, in view of this point, it is assumed that at least one of the shape and luminance of the frame region is controlled based on the content data. Here, the control of the shape of the frame region means that the area of the frame region is also controlled in addition to the control of whether the frame region is rectangular or circular. That is, even when the frame region is rectangular, the area may be large (the frame is thick), or the area may be small (the frame is thin). Further, when the area is set to 0, that is, when the frame region is not set, the shape control of the frame region of this embodiment is included.

  Further, the control of the luminance of the frame area may be control of the luminance value of the display image displayed on the display unit, but is not limited thereto. The above-described problem considered in the present embodiment is caused by the fact that the boundary between the display area and the non-display area in the user field of view becomes clear. Therefore, the brightness of the frame area when the display image is presented to the user ( (Brightness) is important. Here, even if the pixel values of the image data are the same, if the appearance (color and brightness) of the image data for the user is different depending on the hardware control such as the backlight of the display, The brightness of the frame area is not adjusted at the image data (display image) stage, but may be adjusted by hardware control. In other words, the control of the brightness of the frame area in the present embodiment may be adjustment of the brightness value of the frame area portion of the display image, or backlight adjustment in the frame area portion when the display image is displayed. It may be hardware control or both.

  Here, terms used in the present embodiment will be described. Content data is information representing content to be displayed on the wearable display device of the present embodiment. In a narrow sense, the content data is content image information that is an image to be displayed. However, what is included in the content data is not limited to the content image information, and may further include metadata. Here, the metadata is data representing the characteristics of the content, and may be information defining a frame area setting method as will be described later. The content image information may be image data (content image) itself, but is not limited thereto, and may be other information that can specify an image. For example, text data, background color data, and arrangement position information of the text data may be used as the content image information. In this case, the content data provider does not generate the image itself, but the wearable display device generates the content image using the content image information.

  The display image is an image displayed on the display unit of the wearable display device, and is an image generated as a result of the control of the frame area and the like in the present embodiment. The display image includes a content display area and a frame area. In the content display area, the content image represented by the content image information is displayed. In the content display area, when the entire content image is displayed at the same magnification, or when the entire content image is displayed with the scaling process, a part of the content image is displayed with the same magnification or the scaling process. Various examples are conceivable. The frame area is generated by the control of this embodiment, and details will be described later.

  The display area is an area in which the display image in the user field of view is displayed. That is, in the example of FIG. 2B, the entire FIG. 2B is the entire user field of view, and the DA square area in which the text is displayed is the display area. In other words, the display area represents an area in the user field of view, and does not represent a part or the entire area of the image.

  The first to fourth embodiments will be described below as specific examples of the frame region control method. In the first embodiment, a method of controlling the shape of the frame area according to the angle of view (the subject range to be imaged) will be described using an example in which human imaging data is used as content data. In the second embodiment, a method of controlling a frame area based on metadata included in content data will be described using content data related to emergency early warning as an example. In the third embodiment, a method for controlling a frame area based on metadata reflecting a creator's position will be described using an example in which a pre-built movie such as a movie is used as content data, and a content image An example in which a part of the image is displayed in the content display area (an example in which trimming or scaling processing of a content image) is also described. In the fourth embodiment, a content image (displayed in a content display area of a content image in a narrow sense) is used by using an example of a non-prepared video as content data, such as a video by a camera attached to a wearable display device. A method of controlling the frame area based on the luminance value of the portion to be performed will be described. Note that the content data used in the first to fourth embodiments is an example, and it goes without saying that the same processing may be performed using other content data.

2. First Embodiment FIG. 7 shows a configuration example of a wearable display device 10 according to the present embodiment. Wearable display device 10 includes a processing unit 100, a display control unit 200, and a display unit 300. However, the wearable display device 10 is not limited to the configuration shown in FIG. 7, and various modifications such as omitting some of these components or adding other components are possible. For example, although not shown in FIG. 7, a communication unit that acquires content data from an external device or the like, a storage unit that stores the acquired content data or a created display image, a battery for operating each unit, a wearable display A holding unit or the like for holding the device 10 on the user's body may be included. Further, it may include a motion sensor (acceleration sensor or gyro sensor) used for operating the wearable display device 10 or an imaging unit such as a camera. In addition, when an imaging part is included, it is good also considering the image imaged by the said imaging part as content data.

  The processing unit 100 generates a display image by controlling the frame area based on the acquired content data. By displaying the display image including the frame region on the peripheral edge on the display unit 300, it is possible to suppress a decrease in the visibility of the content due to the conspicuous boundary and the influence of vignetting. The function of the processing unit 100 can be realized by hardware such as various processors (CPU and the like), ASIC (gate array and the like), a program, and the like.

  The display control unit 200 performs display control of the display image. In the following description, the display control unit 200 performs control for displaying the display image created by the processing unit 100, and is assumed to be a driver of the display unit 300, for example. However, as described above, the luminance control of the frame area is not limited to that performed at the display image generation stage in the processing unit, and may be performed by hardware control such as a backlight. In that case, the display control unit 200 performs luminance control of the frame area.

  The display unit 300 displays a display image in accordance with control by the display control unit 200. The display unit may be, for example, an eyepiece window provided at the distal end portion of the eyepiece window holding unit in FIG. Since the see-through type wearable display device 10 is assumed in the present embodiment, as a result of the display on the display unit 300, as shown in FIG. The display image is observed as superimposed.

  Next, specific processing of this embodiment will be described. Here, data obtained by imaging a person is used as the content data. In the present embodiment, the shape of the frame region is controlled based on the range (view angle) of the imaged subject.

  First, the entire display mode when the angle of view is wide will be described. In the whole display mode, as is easy to understand when considering the group photo of FIG. 8A, notable content (in this case, a person) may exist not only in the central portion of the content image but also in the region near the peripheral portion. High nature. Therefore, a process of superimposing rectangular frames is performed so that the widest possible range can be seen while alleviating the unnaturalness of the boundary. Unnaturalness becomes more prominent when the brightness of the display content is high and the display is reflected on a mirror frame, but unnaturalness can be reduced by overlapping the frames. Specifically, a frame area FR is set (superimposed in a narrow sense) on the content image, and an area inside the frame area is set as a content area CO. By displaying the display image DI including the frame area FR and the content area CO in the display area (such as DA in FIG. 2B), the influence of the boundary BO is suppressed.

  Examples of specific frame regions are shown in FIGS. 8A to 8C. As shown in FIG. 8A, the frame area may be a single color, but may give the impression that the display area is simply reduced. Therefore, as shown in FIG. 8B, it is desirable to provide a gradient so that the luminance decreases toward the boundary. As the thickness (area) of the frame area here, for example, an area having a size corresponding to 5% of the width of the display area is set, but numerical values can be modified. As shown in FIG. 8B, the process of providing a gradient in the frame region prepares gradient data in which a gradient of luminance is applied in the radiation direction from the center of the display region toward the boundary, and a bit of the gradient data and content data is prepared. It can be realized by a process of taking a logical sum for each. Here, the process of taking the logical sum for each bit is, for example, comparing the brightness values in the grace case of the gradient data and the content data, and the stronger one (here, the brightness value is smaller, so the brightness value is smaller). This is a process of setting the value of (B) to the luminance value of the corresponding pixel. Alternatively, the frame area may be set by a process of blending the luminance values of the gradient data and the content data for each bit.

  The brightness of the frame area is not limited to this, and the frame area shown in FIG. 8C in which a blur such as a Gaussian filter is applied to the border area with the content area is used for a frame area with a single color or a brightness gradient. May be. The frames to be overlapped are assumed to be generated by the processing unit at the time of execution. However, one or more frames to be used are stored in a memory (or a storage unit not shown in FIG. 7) in the processing unit. It may be switched depending on the content.

  Next, an enlarged display mode in which the angle of view is narrower than that of the entire display mode will be described. For example, consider a case where the user enlarges to confirm an individual in the group photo, for example, his / her own face. When the angle of view is narrow in this way, as shown in FIG. 9, there is a high possibility that the content to be noticed is concentrated at the center of the content image. Therefore, as shown in FIG. 9, in order to make the boundary appear more natural, a circular or elliptical frame is overlapped so as not to overlap the recognized face region. The square frame has an impression that the boundary is reduced and the relaxation of the unnaturalness is weak. However, by making the frame into a circular or elliptical shape, it is possible to feel more natural. Furthermore, by not displaying content that may cause vignetting in the vicinity of the apex of the display area, it is possible to reduce user stress due to content vignetting.

  That is, in this embodiment, based on the content data (content image represented by the content image information in a narrow sense), it is determined whether the mode is the entire display mode or the enlarged display mode, and the selected mode is selected. Control is performed to set a frame area corresponding to.

  Here, the determination as to whether the display mode is the whole display mode or the enlarged display mode may be performed by human face recognition processing. As shown in FIGS. 8A to 8C, when a plurality of faces are recognized, it can be determined that the entire display mode is set. Also, as shown in FIG. 9, when the number of faces recognized by enlarging and displaying an individual becomes one, it can be determined that the display mode is the enlarged display mode. In this recognition process, instead of simply determining the number of recognized faces, a specific individual face, for example, one's own face is registered, and the enlarged display mode is used only when the face is recognized. It may be determined that The enlargement process for changing the content image from the group photo state to the personal photo state is performed manually by the user (for example, by searching for a face image of a specific person such as himself and manually selecting an enlargement range). However, for example, a specific person may be automatically searched from the group photo, and when the specific person is detected, the system may automatically perform enlargement display.

  FIG. 10 shows a flowchart for explaining the processing of this embodiment. When this process is started, content data corresponding to the content to be displayed is first acquired (S101). Then, a mode determination process is performed based on the content data (S102). This process is a process of determining whether the angle of view of the content image is wide or narrow, and in a narrow sense, is a determination using the result of the face recognition process as described above.

  When it is determined that the display mode is the whole display mode by the mode determination, a rectangular frame region is set as shown in FIGS. 8A to 8C (S103), and a display image including the frame region is generated. (S105). On the other hand, when it is determined that the display mode is the enlarged display mode, a circular frame region is set as shown in FIG. 9 (S104), and a display image including the frame region is generated (S105).

  Then, the process which displays the produced | generated display image on the display part 300 is performed (S106). Then, it is determined whether or not the current content data is the final data (S107). If it is not the final data, the process returns to S101 to acquire the next content data. On the other hand, if it is final data, the process is terminated.

3. Second Embodiment In the present embodiment, a method for controlling a frame area when content data that is emergency early warning (such as earthquake early warning) is acquired will be described. Since the system configuration example is the same as that described with reference to FIG. 7 in the first embodiment, a detailed description thereof is omitted.

  In a head-mounted display that is assumed to be always worn, it is not a usage method in which information is obtained by always looking at the display part, but it is important that the information can be seen at any time, but not disturbing when it is unnecessary. In such a case, it is necessary to notify the user about the content that has a high degree of urgency and that is desired to be immediately viewed by the user, so that the user's line of sight is directed toward the display area. .

  For notification of important information arrival, a method using voice or vibration (that is, a non-visual method) can be considered. However, it is possible to use a visual method to control the display image so that it can be recognized even if the user is not conscious of the display unit, assuming that the display unit can always be captured in the field of view. A method of displaying an alert is also conceivable.

  Therefore, in the present embodiment, urgency information, which is information related to importance and urgency, is added to the content data as metadata and distributed, and the frame area is controlled according to the urgency information. When the user quickly recognizes highly urgent information such as earthquake bulletin, the visibility by overlapping frames is not so important. Therefore, for information with a high degree of urgency, first, in order for the user to recognize the arrival of information, control is performed so that the area of the frame area is zero, and an alert is displayed on a full screen without a frame. Displaying on the full screen increases the probability of capturing at least a part of the screen in the field of view. For example, as shown in FIG. 11A, it is conceivable to perform alert display by blinking the display area by alternately displaying a full red image and a full white image alternately.

  When the alert display is canceled by the user, a display image in which a frame area is added to the content image representing the specific content of the emergency bulletin is displayed. In this case, since the information to be transmitted to the user has high importance and urgency, the loss of information due to vignetting should be avoided as much as possible. Therefore, here, as shown in FIG. 11B, a display image is generated by setting a frame region so that the content is not chipped. In some cases, processing for reducing the content image (a specific example of this processing will be described later in the fourth embodiment) may be performed so that the content is not crushed by the frame region. Alternatively, when the content image information is composed of text data and the arrangement information of the text data, the wearable display device 10 changes the arrangement of the text data and performs a process of arranging the text data at a position that does not interfere with the frame area. Also good.

  Note that the alert display described above may be canceled by pressing a physically provided button, touching a touch panel, or the like. Alternatively, if the wearable display device 10 includes a sensor or the like so that the user's line-of-sight direction (or the direction the face is facing) can be detected, the alert display is canceled by determination processing based on sensor information or the like. May be.

  FIG. 12 shows a flowchart for explaining the processing of this embodiment. When this process is started, content data corresponding to the content to be displayed is first acquired (S201). Then, it is determined whether metadata corresponding to the urgency information is added to the content data (S202). When the degree of urgency is low, normal display is performed (S210). Here, the normal display may be, for example, the display process described using S102 to S106 in FIG. 10 in the first embodiment, or may be another process.

  When the degree of urgency is high, the area of the frame region is set to 0 (S203), and an alert display for alerting the user is performed (S204). For example, this may be the blinking display shown in FIG. Then, it is determined whether or not the alert is released (S205). If the alert is not released, the process returns to S204 and the alert display is continued. The determination of canceling the alert in S205 may be performed, for example, by detecting the holding position of the eyepiece window holding unit or detecting the line of sight. In addition, the release of the alert may be detected by detecting a specific gesture with a camera or manually pressing a switch.

  When the alert is canceled, the process proceeds to a process of presenting specific information regarding the emergency bulletin. First, a frame region is set to deal with vignetting (S206), and a display image is generated using the frame region as shown in FIG. 11B (S207), and display processing is performed (S208).

  After the processing of S208 or S210, it is determined whether the current content data is the final data (S209). If it is not the final data, the processing returns to S201 to acquire the next content data. On the other hand, if it is final data, the process is terminated.

4). Third Embodiment In this embodiment, a frame area control method when content data that strongly reflects the intention of a producer such as a movie is described. Since the system configuration example is the same as that described with reference to FIG. 7 in the first embodiment, a detailed description thereof is omitted.

  For content with a clear intention of the producer, as shown in FIG. 13, the producer can predetermine the frame shape, brightness, etc. according to the scene or cut. Here, in particular, a movie is considered as content. For example, when the scene 1 is a landscape photographed at a wide angle so that the entire scene can be seen, a square frame with a narrow width is used, and when the scene 3 is a scene where characters are to be highlighted, a circular shape is used. Or use an oval frame. Further, the frame shape may be specified in detail. As for the brightness, when the scene 2 is a night scene, the brightness value of the entire screen is low, so the frame is not used, or when the scene 3 is a daytime scene, the content is high. It is possible to make adjustments such as increasing the frame area so that the luminance values of the screen and the frame change smoothly. Also, for example, a scene type such as landscape, conversation, and action may be designated, and a frame preset by the scene may be used. Alternatively, the user may perform processing for setting a scene type and a frame area corresponding to the type.

  In the above-described processing, it is assumed that metadata for designating a frame area or a scene type is added to each scene by the producer. The wearable display device 10 may use a method of determining a frame region by performing scene recognition processing or the like for each scene. However, since the creator's intention is clear here, the form of the metadata is determined in advance. It is natural to add information with. In this case, the metadata may be information that specifically designates the frame area, or only the type of the scene may be designated, and the frame area corresponding to the type may be selected by the wearable display device 10.

  In the above description including the first and second embodiments, it is assumed that a display image is generated by adding a frame region to the entire content image. However, the present invention is not limited to this, and a display image may be generated by extracting a part of the content image and adding a frame region to the region.

  This example is shown in FIGS. 14 (A) to 14 (C). As shown in FIG. 14A, the original content image is a wide-angle image including not only a person but also a landscape. However, it is conceivable to designate an area that is absolutely desired to be displayed so that the content portion that is the center of the scene is not blocked by the frame. For example, in the case of FIG. 14A, adding a frame area to the entire content image may cause a person's face to be crushed. Accordingly, as shown in FIG. 14B, information for setting a notable area of the content image (in this case, information indicating the screen center CE is not limited to this) is acquired as metadata, and the metadata is acquired. May be used to generate a display image as shown in FIG.

  As shown in FIG. 14C, the display image DI is generated using a part of the content image centered on the screen center CE, so that the resolution of the generated display image DI becomes the resolution of the display unit 300. There is a possibility that it will be low. For example, if the resolution of the entire content image in FIG. 14A matches the resolution of the display unit 300, the display image DI in FIG. 14C is smaller than that. Therefore, the scaling process (enlargement process) may be performed when the display image DI of FIG.

  In addition, the region of interest in the content image (determined by the screen center CE in the above example) may be defined as coordinates on the screen predetermined by the producer, but may be determined by other methods. Good. For example, an area where the movement of the subject is large in the content image can be considered as an important area in the content image. Therefore, as shown in FIG. 15A, motion information MV (motion vector in a narrow sense) is detected from the content image, and the image center CE is detected as shown in FIG. 15B from the center of gravity of the end point of the motion vector. May be defined. The display image DI in this case is as shown in FIG.

  In addition, there is a case where a region of interest can be set in a content image even if the video does not strongly reflect the creator's intention as described above. For example, it is a case where the camera data is added to the wearable display device 10 and the shooting data or the like when shooting is performed as content data. With wearable cameras, it is considered common to shoot everyday scenes, and it seems that family members such as children and pets are taken. In that case, there is a clear subject of interest. Accordingly, a method of tracking (tracking) the subject of interest, setting a region of interest in the content image so that the subject always comes to the center of the screen, and superimposing a frame region on the region of interest may be used. As a result, it is possible to always capture the subject of interest in the center of the screen, and to appreciate while improving the visibility by the frame. For example, as shown in FIGS. 16A and 16B, when there is a subject of interest moving from the upper left of the image toward the lower right of the image on the content image, FIG. By generating the display images DI as shown in FIG. 16B, it is possible to continue to capture the subject of interest at the center of the image. Note that detection of a subject to be tracked may be performed based on face recognition. Alternatively, the user may determine reference data by designating a frame serving as a base point and its position, and perform matching processing using the reference data in subsequent frames.

  Note that the methods of FIGS. 15A to 15C and FIGS. 16A to 16B generate the display image DI from a part of the attention area of the content image. (A) to FIG. 14 (C) are common and have been described in the present embodiment, but differ from the method of FIG. 13 in that metadata defining a frame region is not added. .

  FIG. 17 shows a flowchart for explaining the processing of this embodiment. When this process is started, content data corresponding to the content to be displayed is first acquired (S301). Then, based on the metadata included in the content data, the scene type of the content image is determined (S302), and a frame area corresponding to the scene type is selected (S303). Since the processing after the frame region selection (S304 to S306) is the same processing as S105 to S107 in FIG. 10, detailed description thereof is omitted.

5. Fourth Embodiment In the present embodiment, a method for controlling a frame area when content data that does not strongly reflect the intention of the creator (that is, it is difficult to set a frame area in advance) will be described. Specifically, this is a method of controlling the frame region using the luminance value of at least a part of the content image. Since the system configuration example is the same as that described with reference to FIG. 7 in the first embodiment, a detailed description thereof is omitted.

  A configuration that implements a wearable camera together with a head-mounted display makes sense from the unexpectedness of unconscious continuous video and the expectation of functions as input interfaces such as character recognition and gesture recognition. When viewing wearable camera images on a head-mounted display, it is difficult to determine a frame shape for each scene like a movie. Therefore, for example, as shown in FIG. 18A, the average luminance value of a partial area such as the entire screen or the peripheral area of the screen is calculated, and the luminance average value and the frame are displayed as shown in FIG. The brightness of the frame area may be set so as to match the brightness value. When applying a luminance gradient, as shown in FIG. 18B, the boundary between the frame region and the content region may be an average value, and the gradient may be added so that the luminance decreases as the screen boundary is approached. The value used as the reference for the luminance of the content image may be the maximum value or the minimum value instead of the average value, or may be a value corresponding to 80% of the distribution, and various modifications can be made.

  The area of the frame (width in the case of a rectangular frame) may be fixed, and the luminance gradient may be adjusted to maintain a constant gradient. For example, when the average luminance of the content is calculated and becomes 0xCC as shown in FIG. 19A, the gradient of the luminance gradient of 0x33 set in FIG. As shown in FIG. 19B, a frame in which the luminance changes in four stages starting from 0xCC may be set. In this case, since the gradient of the luminance gradient is the same in FIG. 18B and FIG. 19B, the area (width) of the frame is increased in FIG. 19B compared to FIG. 18B. Yes.

  On the other hand, the luminance gradient may be changed while keeping the area of the frame region constant. In this case, in FIG. 18B, the frame region may be changed in luminance in three steps, whereas in FIG. 19B, four steps of luminance change are required. Therefore, the gradient of the luminance gradient is larger in FIG. 19B than in FIG. 18B.

  In the above method, the content (for example, the subject being imaged) is ignored, and the frame area is controlled only by the luminance, so the scene to be viewed may be hidden by the frame area. For example, as a result of adding a frame region to the content image in FIG. 18A, the subject at the periphery of the content image is crushed as shown in FIG. Therefore, the size of the entire content may be changed according to the frame area. For example, as shown in FIG. 20B, the content image in FIG. 18A may be reduced and a frame area may be added to the reduced image. This processing can be considered to be that a frame area is added so as to overlap the margin area, assuming that the content image in FIG. 18A is reduced and a margin area is generated outside the content image. Alternatively, it can be considered that the reduction processing of the entire image after the addition is performed after adding the frame region to the outside of the content image in FIG. 18A (so as not to overlap the content image). By generating the display image of FIG. 20B, it is possible to always check the entire content regardless of the shape of the frame region.

  FIG. 21 shows a flowchart for explaining the processing of this embodiment. When this process is started, content data corresponding to the content to be displayed is first acquired (S401). Then, the average luminance of the content image is calculated based on the content image included in the content data (S402), and a frame area corresponding to the calculated average luminance is set (S403). Since the processing after setting the frame region (S404 to S406) is the same as S105 to S107 in FIG. 10 and the like, detailed description thereof is omitted.

6). Specific Example of the Present Embodiment In the present embodiment described above, the wearable display device 10 has at least the shape and luminance of the frame area at the periphery of the display image based on the content data to be displayed, as shown in FIG. A processing unit 100 that controls one side to generate a display image including a frame region and a content display region that displays content data, a display control unit 200 that controls display of the generated display image, and a display control unit Based on the display control 200, the display unit 300 includes a display unit 300 that displays the display image generated by the processing unit 100 in a see-through manner so as to overlap the outside world.

  As a result, it is possible to set a frame area according to the content data, thereby reducing the uncomfortable feeling caused by the boundary between the display area and the non-display area in the user's field of view, and suppressing content missing due to vignetting. Thus, it becomes possible to improve the visibility of the content. At that time, since the frame area is controlled based on the content data, the frame area can be controlled appropriately. For example, the circular frame region shown in FIG. 9 has an advantage that the effect of reducing the uncomfortable feeling is higher than that of the square frame region, but has a disadvantage that the area is large and the content display region is easily narrowed. That is, since it is difficult to think of a highly versatile frame area that can be applied to all content data, it is preferable to determine which element should be emphasized based on the content data and change the frame area as appropriate.

  In addition, the content data may include first content data and second content data different from the first content data, and the processing unit 100 may display the first display image corresponding to the first content data. You may set the area | region where at least one of a shape and brightness | luminance differs by a 1st frame area | region and the 2nd frame area | region of the 2nd display image corresponding to 2nd content data.

  Thereby, according to the difference of content data, it becomes possible to set the area | region where at least one of a shape and a brightness | luminance differs also in a frame area | region. Note that the determination of content data (determination of whether or not a plurality of content data has the same characteristics, or how much the characteristics differ if the characteristics are different) is not limited to using one viewpoint, These viewpoints may be combined. For example, as described above, a plurality of determination criteria such as an angle of view at the time of content image capturing, a luminance average value, and metadata can be considered. is there.

  In addition, the processing unit 100 controls the shape so that the area of the frame region is zero for at least one of the first display image and the second display image, and displays the area of the frame region as zero. A display image that does not include the frame area and displays the content display area as a whole may be generated as the image.

  Thereby, as control of the shape of the frame region, it is possible to control the entire surface of the display image as the content display region by setting the area of the frame region to zero. This is used, for example, when the user's attention is effectively attracted in a situation where visibility is not a problem, as in the alert display of FIG. 11A of the second embodiment. Alternatively, as described with reference to FIG. 13 and scene 2 of the third embodiment, the brightness of the content image is sufficiently low, and it may be used when there is no problem even if the frame area is not displayed. In the third embodiment, the frame region is controlled based on metadata in which the area of the frame region is 0 (or metadata indicating that the scene is a sufficiently dark scene). As in the embodiment, the brightness average value of the content image is obtained, and when the value of the brightness average value is smaller than a given threshold, control such as setting the area of the frame region to 0 may be performed.

  Further, when the second content image information included in the second content data is an image obtained by enlarging a part of the first content image information included in the first content data, or the second content image information Is smaller than the angle of view of the first content image information, the processing unit 100 compares the second display image with the area of the first frame region included in the first display image. The area of the second frame region included in the may be set large.

  As a result, as described above in the first embodiment, the frame area can be controlled according to the degree of enlargement or the angle of view of the content image information (content image represented by the content image information in a narrow sense). become. Specifically, in the enlarged display mode, a subject to be noticed is likely to be unevenly distributed in the center of the image, and a circular frame region having a large area is set as shown in FIG. On the other hand, in the entire display mode, since there is a high possibility that the subject of interest is also present in the peripheral portion of the image, a rectangular frame region may be set as shown in FIGS. 8 (A) to 8 (C).

  The processing unit performs a recognition process for recognizing a human face area from the first content image information and the second content image information, and the second content image information is converted into the first content image based on the recognition process. You may perform the determination process whether it is an image which expanded a part of information.

  Accordingly, it is possible to determine whether the display mode is the enlarged display mode or the entire display mode based on the human face recognition process. Since various methods are widely known for the face recognition processing, detailed description thereof is omitted. As a specific example of the determination process based on the result of the face recognition process, the enlarged display mode may be set when the number of faces in the image is 1, and the entire display mode may be set when there are a plurality of faces. Alternatively, the relative size of the face size with respect to the image size may be used in the determination process, and whether or not the person is detected after the recognition of whether or not the face is a specific person in the face recognition process. The determination process may be performed in However, the determination of whether or not it is in an enlarged state is not limited to face recognition. For example, if the content data is related to a captured image, it is possible to acquire data such as zoom magnification at the time of imaging as metadata, and therefore processing for determining whether the angle of view is wide or narrow using the metadata May be performed. When determining using the angle of view, information regarding the angle of view can be acquired for each of the first content image information and the second content image information. Therefore, the subject imaged in the first content image information and the subject imaged in the second content image information are not necessarily the same, and may be different subjects.

  Further, the processing unit 100 may control at least one of the shape and the luminance of the frame region based on the luminance of at least a part of the display image including the content display region.

  Thereby, as described above in the fourth embodiment, it is possible to control the frame area based on the luminance of the display image. As described above, the higher the brightness of the display image, the stronger the sense of discomfort with the non-display area in the user's field of view. Therefore, the effect of reducing discomfort is higher (for example, circular or FIG. 19B). As shown in Fig. 5, a frame area having a large number of gradation gradations is set. In the fourth embodiment, the entire content image is used for luminance calculation, but the present invention is not limited to this. Essentially, the luminance difference between the display area and the non-display area in the user's field of view is a problem, so it is not necessary to use the luminance of the portion of the content image that is not included in the display image (of which the content display area) . Therefore, the attention area that can be included in the display image is determined among the content images as shown in FIGS. 14C and 15C, and the frame area is set from the luminance as in the fourth embodiment. In this case, it is desirable to control the frame area based on the determined luminance in the attention area. Furthermore, in the embodiment as shown in FIGS. 18A to 19B, the peripheral area of the content image has a small contribution to the display image by overlapping the frame area. That is, instead of processing using the entire content image equally, the contribution of the content display area (area that is not collapsed by the frame area) may be increased. However, since the shape and area of the frame region are indefinite at the stage of luminance calculation, and the content display region is not yet determined, it is fully possible to calculate the luminance strictly using only the content display region. May be difficult to implement. In such a case, a technique may be used in which a weighted average in which the weight of the peripheral portion is smaller than that of the central portion is used as the luminance average (in this case, the central portion and the peripheral portion may not coincide with the content display area). However, the entire content image may be handled equivalently as in the fourth embodiment.

  Further, the processing unit 100 may perform control for making the frame region a square or a circle as the control of the shape of the frame region.

  Accordingly, as described in FIGS. 8A to 8C and FIG. 9 of the first embodiment or FIG. 13 of the third embodiment, scene 1 and scene 3, according to the content data. It is possible to selectively use a rectangular frame region and a circular frame region. In general, a circular frame region has an advantage that an effect of reducing a sense of discomfort is high, and a rectangular frame region has an advantage that a content display region occupied in a display image can be easily widened. Therefore, it is determined from the content data which one should be emphasized, and an appropriately shaped frame region is used.

  In addition, the processing unit 100 may perform control of the frame region in a gradation display in which the luminance of the frame region monotonously decreases in the direction from the central portion to the peripheral portion of the display image.

  As a result, as shown in FIG. 8B, it is possible to set a frame region for performing gradation display. It is possible to combine a content display area (display area central portion) with a relatively high luminance and a non-display area (external field visual field) with a relatively low luminance with a smooth luminance change. Compared to a single-color frame region like this, it is possible to improve the effect of reducing discomfort.

  The processing unit 100 may generate a display image by performing blurring processing at the boundary between the frame area and the content display area.

  As a result, as shown in FIG. 8C, the frame region can be set by performing the blurring process. Also in the blurring process, as in the case of gradation display, it is possible to improve the effect of reducing the uncomfortable feeling as compared with the single-color frame area as shown in FIG.

  Further, the processing unit 100 may perform at least one of a size change process and a trimming process on the content image information included in the content data based on the inner diameter of the frame region.

  Thereby, after the shape (area) of the frame region is determined, the size of the content image information (content image in a narrow sense) can be changed. As shown in FIG. 19B, depending on the setting method of the frame area, the area of the frame area with respect to given content data increases, and as a result, the peripheral area of the content image is added by adding the frame area. May collapse greatly. As shown in FIG. 9, there is no problem as long as there is a high possibility that there is no subject of interest at the periphery of the content image, but if this is not the case, the lack of content due to the frame region cannot be ignored. In that regard, as shown in FIG. 20B, by reducing the content image in advance, if many of the content images (the whole in a narrow sense) are displayed in the content display area, the lack of content can be suppressed. it can. Note that the size change processing and trimming processing for the content image here are not limited to displaying the entire content image in the content display region, and may be processing such as trimming the region of interest.

  The processing unit 100 may generate a display image by determining a region of interest in the content image information included in the content data and adding a frame region to the determined region of interest.

  As a result, as shown in FIGS. 14A to 16B, a part of the content image is taken out, the frame area is set so that the area corresponds to the content display area, and the display image is displayed. Can be generated. Therefore, it is possible to effectively improve the visibility of the portion that should be noted in the content image. Note that the region of interest is not limited to the region set as the region, and may be set by designating the center of the image as shown in FIG.

  Further, the processing unit 100 determines a region of interest in the content image information based on the metadata included in the content data, and generates a display image by adding a frame region to the determined region of interest. May be.

  This makes it possible to determine the attention area in the content image based on the metadata. For example, as described above in the third embodiment, a region of interest may be set by a creator of content data, and the information may be acquired as metadata.

  In addition, the processing unit 100 detects the movement of the subject on the content image information, or the result of the subject tracking process that detects the position of the target subject on the content image information. A display image may be generated by determining a region of interest and adding a frame region to the determined region of interest.

  Accordingly, as described above with reference to FIGS. 15A to 16B in the third embodiment, the attention area in the content image can be determined based on the motion detection process or the subject tracking process. It becomes possible. If the subject of interest is known, subject tracking processing for searching for the subject of interest in the content image may be performed, and a region including the detected subject of interest may be set as the region of interest. Even if it is unknown which subject is the target subject, it can be considered that a subject with a large movement in the image has a high degree of attention, and thus the target region can be determined from the motion detection process.

  Further, the content data may include urgency level information indicating the urgency level, and the processing unit 100 may control at least one of the shape and brightness of the frame region based on the urgency level information.

  As a result, as described above with reference to FIGS. 11A and 11B in the second embodiment, it is possible to appropriately display emergency bulletins and the like. The see-through type wearable display device 10 has an advantage that observation of the external field of view is not hindered unless the line of sight is consciously directed to the display area. On the other hand, this advantage involves a risk that the user is not aware of information with a high degree of urgency. Therefore, here, it is assumed that the alert is displayed by a visual method to direct the user's consciousness toward the display area, and the emergency early warning is presented after the consciousness has turned. Specifically, since it is important to make the display area conspicuous during alert display, the area of the frame area may be zero. On the other hand, since it is necessary to avoid missing content as much as possible when presenting information, it is preferable to set a frame region in consideration of vignetting and the like.

  Further, the processing unit 100 may control the display mode of the content data based on the urgency level information.

  As a result, not only the frame area but also the display mode of the content data can be controlled. For example, the blinking cycle in the alert display may be changed according to the urgency level, or the content image layout is changed (if the urgency level is equal to or higher than a certain level, lack of content is not allowed) , Scaling processing and text position adjustment).

  The wearable display device 10 includes an eyepiece window that emits a light beam that forms a display image to be displayed on the display unit 300 toward the user, and the eyepiece window has a width of a projection cross section in the user's visual axis direction. May be 4 mm or less.

  This makes it possible to realize a pupil-division see-through wearable display device as a see-through wearable display device.

  The display control unit 200 may control the brightness of the frame region by controlling the display mode of the display image. Specifically, the display control unit 200 may control the luminance of the frame region by controlling the backlight of the display unit 300.

  As a result, the brightness of the frame area can be controlled by hardware control (backlight control in a narrow sense) when displaying the display image, instead of controlling the brightness value of the frame area in the display image. become. Note that both the processing unit 100 and the display control unit 200 may control the luminance of the frame region.

  Note that the wearable display device 10 of the present embodiment may realize part or most of the processing by a program. In this case, the wearable display device 10 of the present embodiment is realized by a processor such as a CPU executing a program. Specifically, a program stored in a non-temporary information storage medium is read, and a processor such as a CPU executes the read program. Here, the information storage medium (computer-readable medium) stores programs, data, and the like, and functions as an optical disk (DVD, CD, etc.), HDD (hard disk drive), or memory (card type). It can be realized by memory, ROM, etc. A processor such as a CPU performs various processes of the present embodiment based on a program (data) stored in the information storage medium. That is, in the information storage medium, a program for causing a computer (an apparatus including an operation unit, a processing unit, a storage unit, and an output unit) to function as each unit of the present embodiment (a program for causing the computer to execute processing of each unit) Is memorized.

  As mentioned above, although four Embodiments 1-4 and its modification to which this invention was applied were demonstrated, this invention is not limited to each Embodiment 1-4 and its modification as it is, An implementation step The constituent elements can be modified and embodied without departing from the spirit of the invention. Moreover, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above-described first to fourth embodiments and modified examples. For example, you may delete a some component from all the components described in each Embodiment 1-4 and the modification. Furthermore, you may combine suitably the component demonstrated in different embodiment and modification. In addition, a term described together with a different term having a broader meaning or the same meaning at least once in the specification or the drawings can be replaced with the different term anywhere in the specification or the drawings. Thus, various modifications and applications are possible without departing from the spirit of the invention.

DA display area, PU pupil, HO holding part, NA non-display area, BO boundary part,
DI display image, FR frame area, CO content area, CE screen center,
MV motion information, 10 wearable display device, 11 HMD,
100 processing unit, 200 display control unit, 300 display unit

Claims (19)

The display including the frame area and the content display area for displaying the content data by controlling at least one of the shape and brightness of the frame area set at the periphery of the display image based on the content data to be displayed A processing unit for generating an image;
A display control unit that performs display control of the generated display image;
Based on the display control of the display control unit, a display unit that displays the display image generated by the processing unit on the outside world in a see-through manner;
A wearable display device comprising: In claim 1,
The content data includes first content data and second content data different from the first content data,
The processor is
The shape and the luminance of the first frame area of the first display image corresponding to the first content data and the second frame area of the second display image corresponding to the second content data A wearable display device characterized in that at least one of the areas is set in a different area. In claim 2,
The processor is
As the display image in which the area of the frame region is controlled to be zero with respect to at least one of the first display image and the second display image, and the area of the frame region is zero A wearable display device that generates the display image in which the content display area is entirely displayed without including the frame area. In claim 2 or 3,
When the second content image information included in the second content data is an image obtained by enlarging a part of the first content image information included in the first content data,
Alternatively, when the angle of view of the second content image information is narrower than the angle of view of the first content image information,
The processor is
A wearable display characterized in that an area of the second frame region included in the second display image is set larger than an area of the first frame region included in the first display image. apparatus. In claim 4,
The processor is
A recognition process for recognizing a human face area from the first content image information and the second content image information is performed, and the second content image information is converted into the first content image information based on the recognition process. A wearable display device that performs a process of determining whether or not the image is an enlarged image of a part of the image. In any one of Claims 1 thru | or 5,
The processor is
A wearable display device that controls at least one of the shape and the luminance of the frame region based on at least a luminance of the display image including the content display region. In any one of Claims 1 thru | or 6.
The processor is
A wearable display device characterized in that as the control of the shape of the frame region, control is performed so that the frame region is square or circular. In any one of Claims 1 thru | or 7,
The processor is
A wearable display device that controls the frame area to be a gradation display in which the luminance of the frame area monotonously decreases in the direction from the center to the periphery of the display image. In any one of Claims 1 thru | or 7,
The processor is
A wearable display device that generates the display image by performing blurring processing at a boundary between the frame region and the content display region. In any one of Claims 1 thru | or 9,
The processor is
A wearable display device that performs at least one of a size change process and a trimming process on content image information included in the content data based on an inner diameter of the frame region. In any one of Claims 1 thru | or 9,
The processor is
A wearable display comprising: generating a display image by determining a region of interest in content image information included in the content data, and adding the frame region to the determined region of interest apparatus. In claim 11,
The processor is
Based on the metadata included in the content data, the region of interest in the content image information is determined, and the frame region is added to the determined region of interest to generate the display image. A wearable display device characterized by that. In claim 11,
The processor is
Based on the result of the motion detection process that detects the movement of the subject on the content image information or the subject tracking process that detects the position of the target subject on the content image information, the attention of the content image information A wearable display device that determines a region and generates the display image by adding the frame region to the determined region of interest. In any one of Claims 1 thru | or 13.
The content data includes urgency information indicating urgency,
The processor is
A wearable display device that controls at least one of the shape and the luminance of the frame region based on the urgency information. In claim 14,
The processor is
A wearable display device that controls a display mode of the content data based on the urgency information. In any one of Claims 1 thru | or 15,
An eyepiece window for emitting a light beam forming the display image to be displayed on the display unit toward a user;
The eyepiece window is
A wearable display device, wherein a width of a projected cross section in the visual axis direction of the user is 4 mm or less. In any one of Claims 1 thru | or 16.
The display control unit
A wearable display device that controls the brightness of the frame region by controlling a display mode of the display image. In claim 17,
The display control unit
A wearable display device that controls the brightness of the frame region by controlling a backlight of the display unit. Based on the content data to be displayed, at least one of the shape and brightness of the frame region at the periphery of the display image is controlled to generate a display image including the frame region and the content display region for displaying the content data A processing unit to
As a display control unit that performs control to display the generated display image on the display unit in a superimposed manner with the outside world,
A program characterized by causing a computer to function.