JP2015141215A - Head-mounted display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To identify a desired one graphic code by a hands-free, using a head-mounted display device of a simple configuration.SOLUTION: A head-mounted display device 100 comprises: an imaging part 11; a graphic code detection part 271; a virtual line of sight generation part 25; a display part 12; and an identification part 273. The imaging part 11 is configured to be able to photograph a virtual field of view, and the graphic code detection part 271 is configured to detect existence of a graphic code C from a virtual field of view image IM photographed by the imaging part 11. The virtual line of sight generation part 25 is configured to create a virtual line of sight VP in the virtual field of view image IM, and the display part 12 is configured to display the virtual line of sight image IM. The identification part 273 is configured to identify the graphic code C pointed out by the virtual line of sight VP as a specific graphic code SC from the graphic code C in the virtual field of view image IM, and the specific graphic code SC is a graphic code C desired to be decoded.

Description

  The present invention relates to a head mounted display device.

2. Description of the Related Art Conventionally, there is known a head-mounted display device that identifies the position of an identification code such as a bar code written on a tag or the like placed at a predetermined position and reads the identification code whose position is identified.
For example, Patent Literature 1 discloses a head-mounted display that includes display means that causes image light according to image information to enter the user's eyes together with external light and displays an image according to the image information superimposed on the outside scene. ing. The head-mounted display includes imaging means for imaging a user's visual field range, code detection means capable of detecting an invisible identification code existing in the imaging area of the imaging means, and correspondence information associated with the identification code. Display control means for reading correspondence information corresponding to the detected identification code from the stored storage means and performing display control on the display means in correspondence with the identification code. The display control means performs control to display the marker information indicating the position of the identification code at the display position of the display means corresponding to the position of the identification code detected by the code detection means.

  Patent Document 2 discloses a head mounted display that captures at least a part of a user's visual field range and detects the user's hand based on the analysis result. The head-mounted display operates the virtual operation panel based on the detected position of the user's hand as a display position of the virtual operation panel that follows the position of the hand visually recognized by the user. The position of the part is determined and displayed. Further, the head mounted display determines whether or not the virtual operation panel is operated based on the contact position of the user's finger detected by the touch panel, and when it is determined that the virtual operation panel is operated, Control corresponding to the operation position on the virtual operation panel is performed.

JP 2010-210822 A JP 2010-145861 A

  In the head-mounted display disclosed in Patent Document 1, the direction of the line of sight of a user who uses the head-mounted display is detected by a CCD (Charge Coupled Device) camera provided in the head-mounted display. . Thus, in a conventional head mounted display, it is necessary to use a CCD camera or the like when identifying an identification code or the like by detecting the user's line of sight.

  Further, in the head mounted display disclosed in Patent Document 2, a virtual operation panel that follows the position of the user's hand on the position related to the hand visually recognized by the user based on the detected position of the user's hand. The position of the operation unit of the virtual operation panel is determined as the display position. That is, in the head mounted display disclosed in Patent Document 2, it is necessary to indicate the position of the operation unit of the virtual operation panel with a user's finger or the like.

  As described above, in the conventional head mounted display, another means such as a CCD camera is used or the user points to the specific position within the display range of the head mounted display. Therefore, the conventional head-mounted display has a complicated structure, or it is necessary to use a finger to determine a specific position within the display range.

  An object of the present invention is to provide a head-mounted display device that has a simple configuration and can specify a desired one of the graphic codes within a display range in a hands-free manner.

  Hereinafter, a plurality of modes will be described as means for solving the problems. These aspects can be arbitrarily combined as necessary.

  A head-mounted display device according to an aspect of the present invention includes an imaging unit, a graphic code detection unit, a virtual line-of-sight generation unit, a display unit, and a specifying unit. The imaging unit can capture a virtual visual field. The virtual visual field refers to a virtual visual field that is determined based on the position, tilt, and rotation of the photographing unit attached to the worker. The graphic code detection unit detects the presence of the graphic code from the virtual visual field image captured by the imaging unit. The virtual visual field image refers to an image obtained by imaging the virtual visual field captured by the imaging unit. The virtual line-of-sight generation unit generates a virtual line of sight in the virtual visual field image. The virtual line of sight refers to the virtual line of sight of the worker. The display unit is provided at a position where the operator can visually recognize the display unit. Then, the display unit displays a virtual visual field image. The specifying unit specifies the graphic code indicated by the virtual line of sight as the specific graphic code from the graphic code in the virtual visual field image. The specific graphic code is a desired graphic code to be decoded.

  In this head-mounted display device, the virtual visual line generation unit generates a virtual visual line in the virtual visual field image, and the specifying unit specifies the graphic code indicated by the virtual visual line as the specific graphic code from the graphic code in the virtual visual field image. . That is, when the operator moves the imaging unit attached to the operator (that is, adjusts the position, tilt angle, and rotation angle of the imaging unit), the imaging unit determines the virtual field of view determined according to the operation. Take a picture. At this time, the virtual visual line generation unit generates a virtual visual line in the virtual visual field image. Then, the specifying unit specifies the graphic code indicated by the virtual line of sight as the specific graphic code among the graphic codes in the virtual visual field image detected by the graphic code detecting unit. Thus, even if a plurality of figure codes are included in the virtual visual field image, one desired figure code can be specified hands-free.

Therefore, a CCD camera for detecting the line of sight is not required to detect the line of sight of the worker. Also, it is not necessary to point with a finger or the like in order to specify a graphic code in the range indicated in the virtual visual field image.
As a result, the operator can specify one desired graphic code in a hands-free manner using a head-mounted display device having a simpler configuration that does not include a line-of-sight specifying means for specifying the line of sight of the worker.

  The specifying unit may specify the graphic code pointed to by the virtual line of sight as the specific graphic code when the virtual line of sight points to the graphic code for a predetermined time. Thereby, even when the virtual line of sight indicates an undesired graphic code instantaneously or accidentally, it is possible to prevent the graphic code from being erroneously specified.

  The specifying unit may determine that the graphic code within the predetermined distance range is indicated by the virtual visual line when the graphic code exists within the predetermined distance range from the virtual visual line. Thus, the graphic code can be specified as the specific graphic code without adjusting the virtual visual field (virtual visual field image) so that the graphic code and the virtual line of sight overlap. As a result, it becomes easy to perform the specific work of the specific graphic code.

  The virtual visual line generation unit may generate a virtual visual line near the center of the virtual visual field image. This makes it easier for the operator to specify the graphic code using the head-mounted display device.

  The virtual visual line generation unit may generate a virtual visual line in a peripheral region of the virtual visual field image. Thereby, a virtual line of sight can be generated at a position according to the preference of the worker. As a result, the operator can easily perform the graphic code specifying operation using the head-mounted display device.

  The display unit may display the virtual line of sight within the virtual visual field image. Thereby, the worker can confirm at which position the virtual line of sight exists. As a result, the operator can easily perform the graphic code specifying operation using the head-mounted display device.

  The head mounted display device may further include a cancellation unit. The canceling unit cancels the specification of the specific graphic code specified by the specifying unit. Thereby, the operator can cancel specification of the specific graphic code specified once.

  With the head-mounted display device according to the present invention, the position of the graphic code within the display range can be specified hands-free with a simple configuration.

The figure which shows the whole structure of a head mounted display apparatus. The figure which shows the structure of a head mounting part. The figure which shows the structure of an image control part. The figure which shows the basic composition of a figure detection part. The figure which shows the structure of the figure detection part which has a cancellation part. The flowchart which shows the basic operation | movement of a head mounted display apparatus. The flowchart which shows operation | movement of the head mounted display apparatus including the step which erase | eliminates decoding data. The flowchart which shows the specific method of a specific figure code. The figure which shows a mode that the figure code in the virtual visual field image was highlighted. The figure which shows an example of the virtual visual field image with a virtual visual line which attached | subjected the virtual visual line image to the virtual visual field image. The figure which shows the state which the virtual eyes | visual_axis has pointed out the figure code.

1. 1st Embodiment The whole structure of the head mounted display apparatus 100 which concerns on 1st Embodiment is demonstrated using FIGS. 1-2. FIG. 1 is a diagram illustrating an overall configuration of a head mounted display device. FIG. 2 is a diagram illustrating a configuration of the head mounting portion. The head mounted display device 100 is a device that specifies a figure code C (FIG. 6), which is an identification code such as a barcode, in a hands-free manner and decodes the figure code C.

(1) Overall Configuration of Head Mounted Display Device First, the overall configuration of the head mounted display device 100 will be described with reference to FIG. The head mounted display device 100 includes a head mounting unit 1 and an image control unit 2.
The head mounting unit 1 is, for example, a device that is mounted on an operator's head. The head mounting unit 1 captures a region within a predetermined range determined based on the position, tilt angle, and rotation angle of the operator's head, and captures the captured region to the image control unit 2 (described later). Send. In addition, the head mounting unit 1 displays the image after the image processing is performed by the image control unit 2 in the visual field of the operator.

  In this way, a predetermined range determined based on the position of the worker's head, the tilt angle, and the rotation angle is photographed, and the image after the image processing is taken as the virtual field of view image IM (FIG. 6). For example, an HMD (Head Mount Display) can be used as the display device.

The image control unit 2 can be configured as a computer system including a CPU (Central Processing Unit), a storage device, and other interfaces. The image control unit 2 receives the image captured by the head-mounted unit 1 and performs image processing on the received image. And the imaging which performed the image process is transmitted to the head mounting part 1 as the virtual visual field image IM. When image processing is performed in the image control unit 2, the image control unit 2 virtually generates an operator's line of sight in the virtual visual field image IM. This operator's virtual line of sight is referred to as a virtual line of sight VP.
By generating such a virtual visual line VP in the virtual visual field image IM, in the head-mounted display device 100 having a simple configuration that does not include individual means for detecting the visual line of the worker, such as a camera for detecting the visual line, the worker Can be defined.

Further, the image control unit 2 detects one or more graphic codes C in the virtual visual field image IM, and decodes the detected graphic codes C. At this time, the image control unit 2 specifies the graphic code C indicated by the virtual visual line VP in the virtual visual field image IM as a graphic code C (specific graphic code SC (described later)) to be decoded, as necessary.
Thus, the operator can specify the specific graphic code SC to be decoded using the head mounted display device 100 in a hands-free manner.

In addition, said head mounting part 1 and image control part 2 may be comprised integrally as the head mounted display apparatus 100, and may be comprised separately.
When the head mounting unit 1 and the image control unit 2 are configured separately, the image control unit 2 may be a portable terminal or a computer that operates as a server.

In the case where the head mounting unit 1 and the image control unit 2 are separate devices, the head mounting unit 1 and the image control unit 2 may be connected by a cable using a cable or the like, and may be wirelessly connected to each other. Signal transmission / reception may be possible.
When the head mounting unit 1 and the image control unit 2 can transmit and receive signals wirelessly, the head mounting unit 1 and the image control unit 2 transmit and receive signals using a wireless LAN or Bluetooth (registered trademark). Can do.

(2) Structure of the head mounting part Next, the structure of the head mounting part 1 is demonstrated using FIG. FIG. 2 is a diagram illustrating a configuration of the head mounting portion. As shown in FIG. 2, the head mounting unit 1 is, for example, an HMD (head mounted display) mounted on the operator's head. The head mounting unit 1 includes an imaging unit 11 and a display unit 12.
The imaging unit 11 captures an area in a range determined by the size of the light receiving surface of the light receiving unit (not shown) provided in the image capturing unit 11, the direction in which the normal line of the light receiving surface is directed, and the like.

  The imaging unit 11 is attached to the upper center of the head mounting unit 1 (above the operator's eyes and at approximately the midpoint between the left and right eyes of the operator). For this reason, the direction in which the normal line of the light receiving surface of the imaging unit 11 faces changes according to the movement of the operator's head (the position of the head, the tilt angle, and the rotation angle). Therefore, the imaging unit 11 can capture an area determined based on the position, tilt (angle), and rotation (angle) of the operator's head.

As described above, in the present embodiment, the imaging unit 11 is above the eyes of the worker and is attached to a substantially middle point between the left and right eyes of the worker. Therefore, the imaging unit 11 can capture an area almost overlapping with the worker's visual field area. Therefore, an area determined based on the position, tilt (angle), and rotation (angle) of the operator's head taken by the imaging unit 11 is referred to as a “virtual visual field”. .
The imaging unit 11 outputs the captured image as an electrical signal. Therefore, for example, a small camera such as a CCD camera can be used as the imaging unit 11.

  Furthermore, the imaging unit 11 may be capable of adjusting an attachment angle or the like in the head mounting unit 1. Thereby, the imaging part 11 can image | photograph the virtual visual field according to an operator's liking.

The display unit 12 is attached within the visual field of the operator. Further, the display unit 12 displays the image transmitted from the image control unit 2. Therefore, the display part 12 can make an operator visually recognize the virtual visual field image (after-mentioned) transmitted from an image control part. As the display unit 12, for example, a small display device such as a small liquid crystal display can be used.
The display unit 12 may be able to transmit external light (see-through) or may not be able to transmit. When the display unit 12 is see-through, the operator can see the background of the display unit 12 on the side opposite to the side of the operator's eyes without removing the display unit 12 from view.

  Further, the display unit 12 may not display all the images shown in the virtual visual field image IM, or may be capable of adjusting the range of the virtual visual field image IM that can be displayed. Accordingly, for example, a desired image can be displayed on the display unit 12 for the operator, for example, when only the specific graphic code SC (to be decoded) is desired to be displayed.

(3) Configuration of Image Control Unit Next, the configuration of the image control unit 2 will be described with reference to FIG. FIG. 3 is a diagram illustrating a configuration of the image control unit. In FIG. 3, the configuration of the head mounting unit 1 (the imaging unit 11 and the display unit 12) is also shown so that the state of signal transmission / reception with the head mounting unit 1 can be understood.
Each element of the image control unit 2 shown below may be realized as a program that operates in the head mounted display device 100 or a computer system that constitutes the image control unit 2, or is realized by a custom IC or the like. May be.
The image control unit 2 includes an imaging conversion unit 21, a display control unit 23, a virtual visual line generation unit 25, a graphic detection unit 27, and a decoding unit 29.

The imaging conversion unit 21 can receive signals from the imaging unit 11 of the head-mounted unit 1. Therefore, the imaging conversion unit 21 can input an image captured by the imaging unit 11 (virtual visual field imaging) and convert the imaging into a virtual visual field image IM.
The display control unit 23 can receive signals from the imaging conversion unit 21. For this reason, the display control part 23 can acquire the virtual visual field image IM. Further, the display control unit 23 can transmit a signal to the display unit 12 of the head-mounted unit 1. For this reason, the display control unit 23 can transmit the virtual visual field image IM to the display unit 12.

  Furthermore, the display control unit 23 can receive signals from a virtual visual line generation unit 25 (described later). Therefore, the display control unit 23 adds the virtual visual line VP (FIG. 7) to the virtual visual field image IM transmitted from the imaging conversion unit 21 as necessary, and displays the virtual visual line image with virtual visual line VIM as the display unit 12. Can be sent to. Thereby, the display unit 12 can display a virtual visual field image (virtual visual field image with virtual visual line VIM) obtained by adding the virtual visual line VP to the virtual visual field image captured by the imaging unit 11. As a result, the operator can confirm at which position in the virtual visual field the virtual line of sight VP is set.

The virtual line-of-sight generation unit 25 can transmit a signal to a figure detection unit 27 (described later). For this reason, the virtual visual line generation unit 25 can transmit information on the virtual visual line VP to the graphic detection unit 27.
The virtual visual line generation unit 25 generates the virtual visual line VP of the worker in the virtual visual field image IM. The virtual visual field image IM is composed of a large number of dots. Therefore, in the virtual visual field image IM, a coordinate system for defining the position of the virtual visual field image IM can be defined. Therefore, the virtual visual line generation unit 25 can generate the virtual visual line VP as a coordinate point (coordinate value) of the coordinate system indicating the virtual visual field image IM, for example.

Alternatively, the virtual visual line generation unit 25 may generate the virtual visual line VP by defining some feature points that characterize the shape of the virtual visual line VP. In this case, the area inside the feature point of the virtual visual line VP becomes the virtual visual line VP. Further, the virtual visual line generation unit 25 may define the virtual visual line VP by a predetermined number of inequalities. In this case, the region represented by the predetermined number of inequalities is the virtual line of sight VP. For example, when the virtual visual line VP is defined by one inequality (xa) ² + (y−b) ² ≦ R ² , the virtual visual line VP represents the coordinates (a, b) in the virtual visual field image IM. It is defined as a circle with a radius R at the center.

Note that the virtual visual line VP can be generated at a predetermined location in the virtual visual field image IM. For example, the virtual visual line VP may be generated in the vicinity (coordinate value) of the virtual visual field image IM, or may be generated in a predetermined position (coordinate value) in a peripheral region other than the center. For this reason, the head mounted display device 100 may include an adjustment unit (not shown) that adjusts the position of the virtual visual line VP.
Since the head mounted display device 100 includes the adjustment unit, the worker can generate the virtual line of sight VP at a position in the virtual field of view according to the preference of the worker. As a result, the operator can easily perform the specific operation of the specific graphic code SC using the head mounted display device 100.

  In addition, as a figure for indicating the position of the virtual visual line VP, a figure having an arbitrary shape such as a circle, a cross, an arrow, or a character may be used. As a result, an arbitrary figure can be used as a mark of the virtual line of sight VP according to the preference of the worker and the necessity for work.

  The figure detection unit 27 can receive signals from the imaging conversion unit 21. Therefore, the graphic detection unit 27 can input the virtual visual field image IM. The graphic detection unit 27 can receive a signal from the virtual visual line generation unit 25. Therefore, the graphic detection unit 27 can input information regarding the virtual visual line VP. The information related to the virtual visual line VP is, for example, the coordinate value of the virtual visual line VP and / or a predetermined number of inequalities.

The graphic detector 27 can detect the presence of the graphic code C from the input virtual visual field image IM. Then, the graphic detection unit 27 can identify the graphic code C pointed to by the input virtual visual line VP (region) in the virtual visual field image IM among the detected graphic codes C. Here, the graphic code pointed to by the input virtual visual line VP (region) is particularly referred to as “specific graphic code SC”.
Details of the structure of the graphic detection unit 27 will be described later.

The decoding unit 29 can receive a signal from the figure detection unit 27. Accordingly, the decoding unit 29 decodes the specific graphic code SC specified by the graphic detection unit 27 and outputs a decoding result.
In the virtual visual field image IM, the specific figure code SC may be inclined at an arbitrary angle with respect to the horizontal direction of the virtual visual field. In such a case, the decoding unit 29 calculates the tilt angle of the specific figure code SC (which may be an angle with respect to the horizontal direction or an angle with respect to the vertical direction in the virtual visual field), and then calculates the tilt angle to the calculated tilt angle. A specific graphic code SC is read along.

(4) Configuration of Graphic Detection Unit Next, the configuration of the graphic detection unit 27 will be described with reference to FIGS. 4A and 4B. FIG. 4A is a diagram illustrating a basic configuration of the figure detection unit. FIG. 4B is a diagram illustrating a configuration of a figure detection unit having a cancellation unit. As illustrated in FIG. 4A, the graphic detection unit 27 mainly includes a graphic code detection unit 271 and a specifying unit 273.
The figure code detector 271 detects the presence of the figure code C from the input virtual visual field image IM. For example, when the color of the graphic code C is different from the background color in the virtual visual field image IM, the graphic code detection unit 271 displays the portion of the virtual visual field image IM to which the color of the graphic code C (or a color close thereto) is attached. It can be detected as a graphic code C.
In addition, the graphic code detection unit 271 may detect the presence of the graphic code C based on various features such as the shape of the graphic code C.

  The specifying unit 273 selects the figure code C indicated by the virtual line of sight VP defined in the virtual view image IM from the figure codes C detected by the figure code detection unit 271 in the virtual view image IM. As specified. For example, when the virtual visual line VP defined in the virtual visual field image IM and the detected graphic code C overlap, the specifying unit 273 specifies the graphic code C as the specific graphic code SC.

  In the present embodiment, the specifying unit 273 specifies the indicated graphic code C as the specific graphic code SC when the virtual visual line VP indicates the graphic code C for a predetermined time (second time (described later)). Thereby, even when the virtual line of sight VP indicates an undesired graphic code instantaneously or accidentally, it is possible to prevent the graphic code from being erroneously specified.

  For example, the specifying unit 273 includes a dot signal indicating the characteristics of the graphic code C (for example, the color or character of the graphic code C) in the region of the virtual visual line VP defined in the virtual visual field image IM. In this case, it can be determined that the figure code C is indicated by the virtual line of sight VP.

As illustrated in FIG. 4B, the graphic detection unit 27 may further include a cancellation unit 275. The cancel unit 275 cancels the specification of the specific graphic code SC specified by the graphic detection unit 27. Thereby, the operator can cancel specification of the specific graphic code SC specified once.
For example, a message for confirming whether or not to cancel the specification of the specific graphic code SC is displayed in the virtual visual field image IM, and the operator uses the virtual line of sight VP to determine whether or not to cancel the specification of the specific graphic code SC. By indicating, the cancellation unit 275 can determine whether or not to perform the specific cancellation of the specific graphic code SC.

For example, the canceling unit 275 displays a dialog box having a message “Do you want to cancel the specification of the specific graphic code SC?” And a “Yes” button and a “No” button on the display unit 12 (in the virtual visual field image IM). ) And the operator uses the virtual line of sight VP to indicate the “Yes” button or the “No” button, so that the operator can determine whether or not to cancel the specific graphic code SC.
Alternatively, the cancellation unit 275 may cancel the specification of the specific graphic code SC by, for example, recognizing the voice signal having the content “cancel the specification of the specific graphic code SC” by the operator.

(5) Operation of Head Mounted Display Device (5-1) Basic Operation Next, the operation of the head mounted display device 100 of the present embodiment will be described. First, the basic operation of the head mounted display device 100 of this embodiment will be described with reference to FIGS. 5A and 5B. FIG. 5A is a flowchart showing the basic operation of the head mounted display device. FIG. 5B is a flowchart showing the operation of the head mounted display device including the step of erasing the decoded data.
When the head mounted display device 100 starts to operate, first, a virtual visual field is picked up by the imaging unit 11 of the head mounting unit 1 based on the position, tilt angle, and rotation angle of the operator's head. Taken as. Next, the virtual visual field imaging is converted into a virtual visual field image IM by the imaging conversion unit 21. Then, the virtual visual field image IM is output to the display control unit 23, attached with a virtual visual line VP as necessary, converted into a virtual visual field image VIM with virtual visual line, and displayed on the display unit 12.

The display control unit 23 may output the virtual visual field image IM as a still image to the display unit 12 or may output it as a moving image.
When the display control unit 23 outputs the virtual visual field image IM as a still image to the display unit 12, for example, the virtual visual field image IM is displayed on the display unit 12 at a long time interval that is not recognized as a moving image by human vision. indicate. Moreover, when displaying the virtual visual field image IM on the display unit 12 as a still image, the display control unit 23 may perform a process of removing the afterimage phenomenon included in the virtual visual field image IM.

  On the other hand, when the virtual visual field image IM is output to the display unit 12 as a moving image, for example, the virtual visual field image IM is displayed on the display unit 12 at a short time interval that is recognized as a moving image in human vision.

After the virtual visual field image IM is displayed on the display unit 12, the graphic detection unit 27 detects the graphic code C in the virtual visual field image IM and specifies the specific graphic code SC from the detected graphic code C ( Step S1). In specifying the specific graphic code SC in step S1, particularly when there are a plurality of graphic codes C in the virtual visual field image IM, the graphic code indicated by the virtual line of sight VP generated in the virtual visual field image IM. C is specified as the specific graphic code SC. Alternatively, when the graphic code C existing within a predetermined range from the virtual line of sight VP is indicated for a predetermined time or more, the graphic code C may be specified as the specific graphic code SC.
Thereby, the operator can select a desired graphic code C to be decoded from the virtual visual field image IM (within the captured virtual visual field image IM range). The method for specifying the specific figure code SC in step S1 will be described in detail later.

After identifying the specific graphic code SC in step S1, the decoding unit 29 decodes the specific graphic code SC (step S2). The graphic code C (specific graphic code SC) in the present embodiment is a bar code, but is not limited thereto, and other identification codes such as a QR code (registered trademark) may be used as the graphic code C.
After executing the decoding of the specific graphic code SC, the decoding unit 29 outputs the decoding result of the specific graphic code SC.

After executing the decoding of the specific graphic code SC in step S2, the head mounted display device 100 confirms whether or not the decoding operation of the specific graphic code SC has been completed (step S3-1).
The head mounted display device 100 displays, for example, a message for ending the decoding operation of the specific graphic code SC and a button for understanding the end of the decoding operation on the display unit 12, and pointing the button with the virtual line of sight VP. The decoding operation of the specific graphic code SC can be completed.

  When the operator understands the end of the decoding of the specific graphic code SC (“Yes” in step S3-1), the head mounted display device 100 stops its operation. On the other hand, when the operator decides to continue decoding the specific graphic code SC (in the case of “No” in step S3-1), the head mounted display device 100 returns to step S1 to specify the specific graphic code SC, The decoding of the specific figure code SC is continued.

  As described above, in the head-mounted display device 100 according to the present embodiment, by specifying the specific graphic code SC using the virtual line of sight VP, the operator uses the head-mounted display device 100 having a simpler configuration. A desired graphic code C can be specified free of charge.

In the operation of the head mounted display device 100, as shown in FIG. 5B, after the decoding of the specific graphic code SC is executed in step S2, a step of canceling the decoding result of the specific graphic code SC is added as necessary. May be. In this case, as shown in FIG. 4B, the graphic detection unit 27 has a cancellation unit 275.
In the operation of the head mounted display device 100 shown in FIG. 5B, after the execution of step S2, in step S3-2, the cancel unit 275 cancels the specification of the specific graphic code SC currently specified and decodes the specific specification. It is confirmed whether or not the reading result of the graphic code SC is erased.

When the canceling unit 275 cancels the specification of the specific graphic code SC and determines that the operator has agreed to cancel the decoding result of the specific graphic code SC whose specification has been canceled ("Yes" in step S3-2) In this case, the cancel unit 275 cancels the specification of the specific graphic code SC and further erases the decoding result of the specific graphic code SC.
Then, the process returns to step S1, and the identification of the specific graphic code SC and the decoding of the specific graphic code SC are continued again.

  In this way, the cancel unit 275 cancels the specification of the specific graphic code SC, and further erases the decoding result of the specific graphic code SC whose specification has been canceled, so that the specific graphic code SC that the specific graphic code SC wants to decode is deleted. If it is not C, the operator can cancel the specification of the specific graphic code SC and erase the decoding result of the specific graphic code SC.

  On the other hand, when the cancellation unit 275 determines that the operator does not agree to delete the decoding result of the specific graphic code SC (“No” in step S3-2), the process proceeds to step S4. In step S4, the graphic detection unit 27 notifies the decoding unit 29 of the successful decoding of the specific graphic code SC, and the decoding unit 29 outputs the decoding result (step S4).

Thereafter, similarly to step S3-1 shown in FIG. 5A, it is confirmed whether or not the decoding operation of the specific graphic code SC is to be ended (step S5). When the operator understands that the decoding operation of the specific graphic code SC is finished (in the case of “Yes” in step S5), the head mounted display device 100 stops its operation.
On the other hand, when the operator does not understand that the specific graphic code SC has been decoded (“No” in step S5), the process returns to step S1, and the head mounted display device 100 continues the operation.

(5-2) Specific Graphic Code Identification Method Next, a specific graphic code SC identification method in step S1 of FIGS. 5A and 5B will be described with reference to FIG. 5C. FIG. 5C is a flowchart showing a method of specifying the specific graphic code SC.
When the head-mounted display device 100 starts specifying the specific graphic code SC, first, the virtual field of view determined by the imaging unit 11 based on the position of the operator's head, the tilt angle, and the rotation angle is used as the virtual field of view imaging. Taken. Next, the virtual visual field imaging is converted into a virtual visual field image IM by the imaging conversion unit 21. The virtual visual field image IM is output to the display control unit 23 and displayed on the display unit 12 (step S101).

  As will be described later, in this embodiment, the virtual visual line VP is generated when there are a plurality of graphic codes C in the virtual visual field image IM, but the generation timing of the virtual visual line VP is limited to this. I can't. The display control unit 23 adds the virtual visual line VP to the virtual visual field image IM as the virtual visual image with virtual visual line VIM before determining whether or not the plurality of graphic codes C are detected in the virtual visual field image IM. Good. Thus, the operator can confirm the position of the virtual visual line VP in the virtual visual field image IM before determining whether or not a plurality of graphic codes C are detected in the virtual visual field image IM.

After the imaging unit 11 captures the virtual visual field imaging, the graphic code detection unit 271 of the graphic detection unit 27 tries to detect the graphic code C from the received virtual visual field image IM. Then, the graphic code detector 271 confirms whether or not the graphic code C is detected (step S102).
When the figure code detection unit 271 does not detect the figure code C (“No” in step S102), the process returns to step S101, and the imaging unit 11 continues to capture the virtual visual field.
Note that when the graphic code detection unit 271 fails to detect the graphic code C, for example, when the graphic code C in the virtual visual field image IM is distorted, the detection fails, or in the virtual visual field image IM. The figure code detecting unit 271 recognizes the figure code C when, for example, the shape or color of a certain object is similar to the figure code C, but it cannot be clearly determined whether it is the figure code C or not. An error notifying the failure may be output.

On the other hand, when the graphic code detection unit 271 detects the graphic code C (“Yes” in step S102), the graphic code detection unit 271 further checks the number of detected graphic codes C (step S103). .
If the graphic code detection unit 271 detects the graphic code C, the display control unit 23 highlights the detected graphic code C. In the present embodiment, as shown in FIG. 6, a graphic code C (the graphic code C shown in FIG. 6 is a barcode) detected in the virtual visual field image IM with a white background is displayed as a conspicuous frame such as a red thick frame. The figure code C is highlighted by a method of enclosing with f. FIG. 6 is a diagram illustrating a state in which the graphic code in the virtual visual field image is highlighted.

  Other highlighting methods of the detected graphic code C include, for example, a method of changing the background color of a portion other than the detected graphic code C area to a color in which the graphic code C is conspicuous, and adding a number to the detected graphic code C. For example, a method for notifying the number of figure codes C detected by voice, or the like.

  When the number of figure codes C detected by the figure code detection unit 271 is 1 (in the case of “No” in step S103), the specifying unit 273 specifies one detected figure code C as the specified figure code SC. (Step S104). Then, the identification of the specific graphic code SC is finished.

In the above description, the specifying unit 273 specifies the figure code C detected unconditionally as the specified figure code SC when the number of detected figure codes C is 1, but the invention is not limited thereto. Even when the graphic code detection unit 271 can detect only one graphic code C, a virtual visual line VP, which will be described below, is attached to the virtual visual field image IM, and the one graphic code C is generated by the virtual visual line VP. When indicated, the one graphic code C may be specified as the specific graphic code SC.
As a result, even if only one graphic code C is detected, it can be confirmed whether or not the graphic code C is the desired graphic code C to be decoded, and the graphic code C can be specified as the specific graphic code SC.

On the other hand, when the number of figure codes C detected by the figure code detection unit 271 is two or more (in the case of “Yes” in step S103), the virtual line-of-sight generation unit 25 generates the virtual line-of-sight VP and the virtual line-of-sight VP. Information regarding the virtual visual line VP including information such as the shape and size of the image is transmitted to the graphic detection unit 27 (step S105). Then, the graphic detection unit 27 adds the virtual visual line VP to the virtual visual field image IM based on the received information regarding the virtual visual line VP.
In step S105, the virtual visual line generation unit 25 transmits information related to the virtual visual line VP to the display control unit 23, and the virtual visual line VP received by the display control unit 23 is added to the virtual visual field image IM. An attached virtual visual field image VIM is generated. Thereby, the display part 12 can display virtual visual field image IM (virtual visual field image VIM with virtual visual line) which attached | subjected virtual visual line VP.

  Here, for example, as shown in FIG. 7, it is assumed that a circular virtual visual line VP is added to a substantially central portion of the virtual visual field image IM in which two graphic codes C exist. FIG. 7 is a diagram illustrating an example of a virtual visual field image with a virtual visual line obtained by adding a virtual visual line to the virtual visual field image.

After adding the virtual visual line VP to the virtual visual field image IM in step S105, the specifying unit 273 determines whether or not the graphic code C in the virtual visual field image IM is indicated by the virtual visual line VP (step S106).
At this time, the specifying unit 273 can determine whether or not the virtual line of sight VP points to the graphic code C by a so-called “hit determination” method used in a computer game or the like, for example. Specifically, for example, the specifying unit 273 determines whether the graphic code C and the virtual code VP are virtual depending on whether or not an image dot indicating the characteristic of the graphic code C exists in the region of the virtual visual line VP indicated in the information about the virtual visual line VP. By determining whether or not the line of sight VP overlaps, it is possible to determine whether or not the virtual line of sight VP indicates the graphic code C.

For example, as shown in FIG. 7, the virtual visual field image IM is shown in FIG. 8 when the worker moves his / her head from the state where the virtual visual line VP and the graphic code C do not overlap in the virtual visual field image IM. Consider the case where changes occur. FIG. 8 is a diagram illustrating a state in which the virtual line of sight points to the graphic code.
At this time, as shown in FIG. 8, the circle of the virtual line of sight VP includes the background color of the graphic code C and a part of the pattern of the graphic code C indicating the characteristics of the graphic code C. . Thus, in the virtual visual field image IM, when the image dot indicating the feature of the graphic code C is included in the region of the virtual visual line VP, the specifying unit 273 determines that the virtual visual line VP and the graphic code C are It is determined that they overlap, and it is determined that the virtual line of sight VP points to the figure code C.

  When the virtual visual line VP points to the graphic code C, the display control unit 23 may further highlight the graphic code C pointed to by the virtual visual line VP. In this case, for example, when the graphic code C is detected in step S102, the display control unit 23 may further highlight using the same highlighting method as the method of highlighting the graphic code C, or the previous highlighting. Further highlighting may be performed using a method different from the display method.

  When the identifying unit 273 determines that the graphic code C is not indicated by the virtual line of sight VP (“No” in step S106), the process proceeds to step S107. On the other hand, when the specifying unit 273 determines that the graphic code C is pointed by the virtual line of sight VP (“Yes” in step S106), the process proceeds to step S108.

In step S107, the graphic detection unit 27 checks whether or not a first time has elapsed since the graphic code detection unit 271 detected the graphic code C in step S102 (step S107).
When the graphic detection unit 27 determines that the first time has not elapsed since the graphic code C was detected (in the case of “No” in step S107), the process returns to step S106, and the graphic code C is represented by the virtual line of sight VP. The detection process of whether or not it is pointed is continued.

On the other hand, when the graphic detection unit 27 determines that the first time has elapsed since the graphic code C was detected (“Yes” in step S107), the process returns to step S101 to acquire the virtual visual field image IM and the graphic Redo code C detection.
In this way, if the graphic code C is not indicated by the virtual line of sight VP even after a predetermined time (first time) has passed since the graphic code C is detected, the graphic code C is detected again and specified. It can suppress that the part 273 performs wasteful determination whether the figure code C is pointed by the virtual visual line VP. As a result, the processing load on the head mounted display device 100 can be reduced.

In step S108, the specifying unit 273 confirms whether or not the second time has elapsed since the graphic code C is indicated by the virtual line of sight VP.
When the specifying unit 273 determines that the second time has not elapsed since it was determined that the graphic code C is indicated by the virtual line of sight VP (in the case of “No” in step S108), the process returns to step S106. The determination of whether or not the graphic code C by the specifying unit 273 is indicated by the virtual line of sight VP is continued.

  On the other hand, when the specifying unit 273 determines that the second time has elapsed since it was determined that the graphic code C is indicated by the virtual line of sight VP (in the case of “Yes” in step S108), the specifying unit 273 Then, the graphic code C pointed to by the virtual line of sight VP is specified as the specific graphic code SC (step S109). Then, the head mounted display device 100 ends the specification of the specific graphic code SC.

  In this way, in step S108, the specifying unit 273 specifies the figure code C pointed to by the virtual line of sight VP after a predetermined time (second time) has elapsed since the figure code C was pointed to by the virtual line of sight VP. By specifying the graphic code SC, for example, the graphic code C other than the desired graphic code C is accidentally indicated by the virtual line of sight VP while searching for the graphic code C that the operator wants to decode. It is possible to prevent the graphic code C other than the graphic code C from being erroneously specified as the specific graphic code SC.

In this way, the virtual visual line VP, which is a virtual visual line, is added to the virtual visual field image IM captured by the imaging unit 11, and in the virtual visual field image VIM with virtual visual line, the graphic code C is transmitted by the virtual visual line VP for a predetermined time. (Second time) When pointed, the pointed figure code C is determined to be the specific figure code SC, so that it is erroneously determined in a hands-free manner using the head-mounted display device 100 having a simpler configuration. The specific graphic code SC can be specified while reducing the probability of.
In the above embodiment, the specific step of specifying the graphic code C pointed to by the virtual visual line VP in the virtual visual field image IM as the specific graphic code SC using steps S106 to S109 has been specifically described. However, the specific step is not limited to the above embodiment.

2. Effects of the First Embodiment The effects of the head mounted display device 100 of the first embodiment can be described as follows. The head mounted display device 100 (an example of a head mounted display device) includes an imaging unit 11 (an example of an imaging unit), a graphic code detection unit 271 (an example of a graphic code detection unit), and a virtual visual line generation unit 25 (virtual visual line generation). Unit), a display unit 12 (an example of a display unit), and a specifying unit 273 (an example of a specifying unit). The imaging unit 11 can capture a virtual visual field (an example of a virtual visual field). The graphic code detection unit 271 detects the presence of the graphic code C (an example of the graphic code) from the virtual visual field image IM (an example of the virtual visual field image) captured by the imaging unit 11. The virtual visual line generation unit 25 generates a virtual visual line VP (an example of a virtual visual line) in the virtual visual field image IM. The display unit 12 is provided at a position that can be visually recognized by the operator. Then, the display unit 12 displays the virtual visual field image IM. The specifying unit 273 specifies the graphic code C indicated by the virtual line of sight VP as the specific graphic code SC (an example of the specific graphic code SC) from the graphic code C in the virtual visual field image IM. The specific graphic code SC is a desired graphic code C to be decoded.

  In the head mounted display device 100, the virtual line-of-sight generation unit 25 generates a virtual line of sight VP in the virtual visual field image IM, and the specifying unit 273 further displays the graphic code indicated by the virtual visual line VP from the graphic code C in the virtual visual field image IM. C is specified as the specific graphic code SC. That is, when the operator moves the head (that is, adjusts the position, tilt angle, and rotation angle of the head), the imaging unit 11 captures a virtual field of view determined according to the operation. At this time, the virtual visual line generation unit 25 generates the virtual visual line VP in the virtual visual field image IM. Then, the specifying unit 273 specifies the graphic code C indicated by the virtual line of sight VP as the specific graphic code SC among the graphic codes C in the virtual visual field image IM detected by the graphic code detecting unit 271. In this way, even if a plurality of graphic codes C are included in the virtual visual field image IM, one desired specific graphic code SC can be specified in a hands-free manner.

Therefore, the head mounted display device 100 does not require a CCD camera for detecting the line of sight for detecting the line of sight of the operator. Further, it is not necessary to point with a finger or the like in order to specify the figure code C in the range indicated in the virtual visual field image IM.
As a result, the operator can specify one desired graphic code C in a hands-free manner using the head-mounted display device 100 having a simpler configuration that does not include the visual line direction specifying means for specifying the visual line of the worker.

  In the above embodiment, the identifying unit 273 identifies the graphic code C pointed to by the virtual visual line VP as the specific graphic code SC when the virtual visual line VP points to the graphic code C for the second time (an example of a predetermined time). doing. Thereby, even when the virtual visual line VP points to an undesired graphic code C instantaneously or accidentally, it is possible to prevent the graphic code C from being erroneously specified.

  In the above embodiment, when the graphic code C exists within a predetermined distance from the virtual visual line VP, the specifying unit 273 indicates that the graphic code C within the predetermined distance is indicated by the virtual visual line VP. Judging. As a result, the graphic code C can be specified as the specific graphic code SC without adjusting the virtual visual field (virtual visual field image IM) so that the graphic code C and the virtual visual line VP overlap. As a result, the specific work of the specific graphic code SC can be easily performed.

  In the above embodiment, the virtual visual line generation unit 25 generates the virtual visual line VP near the center of the virtual visual field image IM. This makes it easier for the operator to specify the graphic code C using the head mounted display device 100.

  As another embodiment, the virtual visual line generator 25 can generate the virtual visual line VP also in the peripheral region of the virtual visual field image IM. Thereby, the virtual visual line VP can be generated at a position according to the preference of the worker. As a result, the operator can easily perform the specific operation of the graphic code C using the head mounted display device 100.

  In the above embodiment, the display unit 12 displays the virtual visual line VP in the virtual visual field image IM. Thereby, the operator can confirm in which position the virtual visual line VP exists. As a result, the operator can easily perform the specific operation of the graphic code C using the head mounted display device 100.

  As another embodiment, the head mounted display device 100 may further include a cancellation unit 275 (an example of a cancellation unit). The cancel unit 275 cancels the specification of the specific graphic code SC specified by the specifying unit 273. Thereby, the operator can cancel specification of the specific graphic code SC specified once.

3. Other Embodiments Although one embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and various modifications can be made without departing from the scope of the invention. In particular, a plurality of embodiments and modifications described in this specification can be arbitrarily combined as necessary.
In the head mounted display device 100 according to the first embodiment, the specifying unit 273 determines that the virtual line of sight VP indicates the graphic code C when the virtual line of sight VP and the graphic code C occupy the same region. It was. However, when the virtual visual line VP and the graphic code C occupy the same region, it is not limited to determining that the virtual visual line VP indicates the graphic code C.

For example, the specifying unit 273 may determine that the virtual line of sight VP indicates the graphic code C even when the graphic code C exists within a predetermined range from (a boundary of) the virtual visual line VP. In this case, for example, the virtual line-of-sight generation unit 25 may define a second region within a predetermined distance from (a boundary of) the virtual line of sight VP and include information on the second region in the information regarding the virtual line of sight VP. Good.
Thus, the operator moves the head so that the virtual line of sight VP approaches the figure code C without adjusting the movement of the head so that the graphic code C to be read and the virtual line of sight VP occupy the same region. The graphic code C can also be specified as the specific graphic code SC by moving it. As a result, the operator can easily perform the specific work of the specific graphic code SC using the head mounted display device 100.

  In the above embodiment, the virtual line of sight VP is displayed in the virtual visual field image IM, but the present invention is not limited to this. The display unit 12 may not display the virtual visual line VP in the virtual visual field image IM. As long as the operator adjusts the virtual visual field so that the desired graphic code C falls within a predetermined range in the virtual visual field image IM, the specifying unit 273 indicates that the virtual line of sight VP indicates the graphic code C. This is because it can be judged. As a result, the graphic code C can be specified as the specific graphic code SC.

  The present invention can be widely applied to head mounted display devices.

DESCRIPTION OF SYMBOLS 100 Head mounted display apparatus 1 Head mounting part 11 Imaging part 12 Display part 2 Image control part 21 Imaging conversion part 23 Display control part 25 Virtual gaze generation part 27 Graphic detection part 29 Decoding part 271 Graphic code detection part 273 Specification part 275 Cancel Part C Graphic code SC Specific graphic code IM Virtual visual field image VIM Virtual visual field image VP with virtual visual line VP Virtual visual line f Frame

Claims (7)

An imaging unit that is attached to an operator and can capture a virtual field of view;
A graphic code detection unit that detects the presence of a graphic code from the virtual visual field image captured by the imaging unit;
A virtual visual line generation unit that generates a virtual visual line that is a virtual visual line of the worker in the virtual visual field image;
A display unit provided at a position where the operator can visually recognize the virtual visual field image;
From a graphic code in the virtual visual field image, a specifying unit that specifies a graphic code indicated by the virtual line of sight as a specific graphic code;
A head mounted display device.   2. The head mounted display device according to claim 1, wherein when the virtual line of sight points to the graphic code for a predetermined time, the specifying unit specifies the graphic code pointed to by the virtual line of sight as the specific graphic code.   The specifying unit determines that a graphic code within the predetermined distance range is indicated by the virtual visual line when the graphic code exists within a predetermined distance range from the virtual visual line. Item 3. The head mounted display device according to Item 1 or 2.   The head mounted display device according to claim 1, wherein the virtual visual line generation unit generates the virtual visual line in the vicinity of the center of the virtual visual field image.   The head mounted display device according to claim 1, wherein the virtual visual line generation unit generates the virtual visual line in a peripheral region of the virtual visual field image.   The head mounted display device according to claim 1, wherein the display unit displays the virtual line of sight within the virtual visual field image.   The head mounted display apparatus in any one of Claims 1-6 further provided with the cancellation part which cancels specification of the said specific figure code specified by the said specific part.

Images