JP2016144071A - Position control device, program, image display device, and image display method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a device and method for, even when a user is viewing any of a plurality of displayed content, enabling the user to take communication by matching his or her visual line.SOLUTION: A position control device for controlling the position of content displayed on the display surface of display means includes: imaging means 14 arranged at a predetermined position on the display surface for imaging a user facing the display surface; reproduction means for displaying a plurality of content on the display surface in accordance with a set condition or instruction; detection means 32 for detecting a position on the display surface being viewed by the user on the basis of a pickup image; determination means for determining the deviation of the visual line of the user relative to the position of the imaging means 14 on the basis of the detected position; and rearrangement instruction means 34 for instructing the change of the arrangement on the display surface of at least the content present at the position being viewed by the user among the plurality of content to the reproduction means such that the position of the imaging means 14 and the position on the display surface being viewed by the user are made coincident or close within a predetermined distance.SELECTED DRAWING: Figure 5

Description

  The present invention relates to a position control device for controlling the position of content displayed on a display surface of a display means, a program for causing a computer to control the position, an image display device including the position control device, and a method thereof.

  With the development of network technology, remote communication using cameras and display means has become widespread. Through this remote communication, for example, employees in a company can make physical health consultations and receive mental health counseling without going to an industrial physician.

  In such health consultations and mental health counseling, the occupational physician communicates with the employee from the perspective of the eye, so that the employee can recognize that he / she is listening to his / her own story and gain a sense of security. . However, remote communication using a camera and a display means has a problem that it is difficult to communicate by matching eyes.

  Therefore, a method of installing a half mirror in front of the camera and projecting the other party's face on the half mirror, or a method in which the display means is a projector, the camera is placed on the display surface, and the other party's face is projected on the projector. (For example, refer to Patent Document 1). In these methods, a camera is arranged on the display surface, and the line of sight is aligned with the camera, so that communication can be performed with the eyes aligned.

  In addition to the image of the other party's face, a plurality of contents such as materials can be simultaneously displayed on the display surface. For this reason, one user can communicate with the other user while viewing the document in addition to the other user's face.

  However, in the conventional method described above, the face image is displayed at the camera position, the document is displayed at other positions, and when the line of sight moves to the document, the line of sight moves away from the camera, and the eyes are aligned to communicate. There was a problem that could not be taken.

  Therefore, it has been desired to provide an apparatus and a method that enable communication by matching the line of sight when viewing any of a plurality of displayed contents.

  In view of the above problems, the present invention is a position control device that controls the position of content displayed on a display surface of a display unit, and is located at a predetermined position on the display surface and images a user facing the display surface. Imaging means, reproduction means for displaying a plurality of contents on a display surface in accordance with set conditions or instructions, detection means for detecting a position on the display surface viewed by a user based on the captured image, and detection Based on the determined position, the determination unit that determines the shift of the user's line of sight with respect to the position of the imaging unit, and the position of the imaging unit and the position on the display surface that the user is viewing match according to the determination result of the determination unit Or an arrangement instruction for instructing the reproduction means to change the arrangement of at least the content at the position viewed by the user on the display surface so as to approach within a predetermined distance And a stage position control apparatus is provided.

  According to the present invention, it is possible to communicate while keeping an eye on any of a plurality of displayed contents.

The figure which showed the structural example of the remote communication system. The figure which showed the hardware constitutions of the image display apparatus. The figure which showed the place which is projecting the image | video of a remote place with the projector as a projection means with which an image display apparatus is provided. The figure which shows the place which also projects the data with the image of the remote place. The block diagram which showed one function structure of the image display apparatus containing a position control apparatus. The block diagram which showed another functional structure of the image display apparatus containing a position control apparatus. The sequence diagram which showed the flow of the 1st process implemented by an image display apparatus. The sequence diagram which showed the flow of the 2nd process implemented by an image display apparatus. The sequence diagram which showed the flow of the 3rd process implemented by an image display apparatus. The figure which showed the 1st example which changes arrangement | positioning of the content currently displayed. The figure which showed the 2nd example which changes arrangement | positioning of the content currently displayed. The figure which showed the 1st example of the position of the eyes | visual_axis on the displayed content. The figure which showed the 2nd example of the position of the eyes | visual_axis on the content currently displayed.

  FIG. 1 is a diagram illustrating a configuration example of a remote communication system. In FIG. 1, it is configured as a system for communicating between two points far away from each other, but is not limited to two points, and may be three or more points. The two points may be one in the country and the other in the overseas, or both in the country and in different regions. The two points may be different buildings on the same site, different floors in the same building, different rooms on the same floor, and the like.

  The remote communication system includes an image display device at each point, and the image display devices are connected to each other via a network 10 such as the Internet. The image display device at the left A point is used by the user A sitting on a chair, and the screen 11A, a projection unit 12A for projecting an image captured at the right B point on the screen 11A, and information processing Means 13A and imaging means 14A are included. Similarly, the image display device at the point B is used by the user B sitting on a chair, and includes a screen 11B, a projection unit 12B, an information processing unit 13B, and an imaging unit 14B. Here, the images of the users A and B are captured by the imaging units 14A and 14B. However, the images are not limited to moving images, and may be still images. In the case of a still image, image data can be acquired and projected at regular time intervals. Hereinafter, since the image display device at the point A and the image display device at the point B are the same device, only the image display device at the point A will be described.

  The screen 11A has a plane (display surface) for projecting an image that is one of the contents to be displayed, and a window is provided at a predetermined position on the display surface. This window is for capturing the light reflected by the user A when the imaging means 14A such as a camera disposed on the back surface of the screen 11A images the user A facing the screen 11A. The screen 11A is provided with a window so as to be positioned between the eyes of the user B displayed on the screen 11A in order to project an image of the upper body of the user B at the point B. As described above, the imaging means 14A is arranged at a position close to the eyes of the user B displayed on the screen 11A, and the conversation can be performed while looking at the eyes of the user B, thereby communicating with each other.

  The projection unit 12A is a projector or the like that projects the image of the user B and other contents such as a medical chart, and the projector may be used in any CRT (Cathode Ray Tube) type or light valve type projector. it can. In recent years, many light valve projectors have been used. There is a liquid crystal projector as a light valve type projector. The liquid crystal projector includes a light source such as a mercury lamp, a liquid crystal panel that generates and outputs an image by transmitting light from the light source, and a magnifying lens that enlarges the output image and projects it onto the screen 11A. Is done. The projection unit 12A includes a communication I / F for connecting to the information processing unit 13A via a cable or a wired connection or infrared communication, Bluetooth (registered trademark), a wireless LAN, or the like.

  The information processing means 13A receives the video data transmitted from the point B, sends the video data to the projection means 12A, acquires the video data of the video imaged by the imaging means 14A, and transmits it to the B point. In addition to the video data, the information processing means 13A acquires audio data and transmits it to the B point, receives the audio data acquired at the B point, and outputs the audio data. The information processing means 13A may be any device that can realize this function. For example, a PC, a tablet terminal, a smartphone, an MFP (Multi-Function Peripheral), a workstation, a PDA (Personal Digital Assistant), or the like can be used.

  The information processing means 13A is equipped with one or more applications, makes it possible to create, save, and open materials such as medical records, sends the materials to the projection means 12A as one of the image data, and projects them on the screen 11A Can be made. The material may be a material that can be shared by two or more users, or may be a material that only an individual refers to. In the case of a shared material, in addition to being sent to the projecting means 12A, it can also be sent to the point B and projected onto the screen 11B for display.

  When the information processing unit 13A projects and displays both the user B's video and material on the screen 11A, the information processing unit 13A also outputs information about the size and position of the user B's video and material to be displayed to the projection unit 12A. . Thereby, the projection unit 12A can project the video and material of the user B on the screen 11A based on the information.

  The imaging unit 14A is a digital camera, a video camera, or the like, and includes a lens for capturing light, a photoelectric conversion element that converts the captured light into an electrical signal, and an A / D converter that converts the electrical signal into digital data. Is done. The photoelectric conversion element may be a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor. In addition, the imaging unit 14A may include an image processing unit that performs white balance, gain control, gamma correction, shading correction, and the like. Further, the imaging unit 14A includes a communication I / F for connecting to the information processing unit 13A through a cable or a wireless connection through a wireless LAN (Local Area Network). The communication I / F may be wirelessly connected by infrared communication or Bluetooth (registered trademark).

  Although FIG. 1 illustrates the configuration using the screen 11A and the projection unit 12A, the present invention is not limited to this. If one or more contents such as the image or material of the user B can be displayed, the display Such display means may be used.

  A hardware configuration of the image display apparatus will be described with reference to FIG. The image display device has the same configuration at point A and point B. For this reason, the description will be made using only numerals that omit the signs “A” and “B” that indicate which point. The image display device includes a CPU 20, ROM 21, RAM 22, HDD 23, communication I / F 24, external storage I / F 25, input / output I / F 26, audio input device 27, audio output device 28, information processing means 13, image pickup means 14, A screen and projection means (not shown) are provided. Each hardware in the information processing means 13 is connected to the bus 29 and can exchange data and the like via the bus 29. Here, the information processing means 13 will be described as a PC.

  The CPU 20 controls the entire PC. The ROM 21 stores a boot program for starting up the PC, firmware for controlling the HDD 23, and the like. The RAM 22 provides a work area for the CPU 20. The HDD 23 is a non-volatile memory, and stores the above-described application executed by the CPU 20 and data of materials. The communication I / F 24 is connected to the network 10 and enables communication with image display apparatuses at other points via the network 10. In addition, the communication I / F 24 is connected to the projection unit and enables communication with the projection unit.

  The external storage I / F 25 is a CD-ROM drive, SD card slot, etc., and is equipped with a recording medium such as a CD-ROM or SD card, and controls the writing of data to the storage medium and the reading from the recording medium. . The input / output I / F 26 is an audio input / output terminal or the like, and is equipped with an audio input device 27 such as a microphone, an audio output device 28 such as a speaker, and the like, and controls input / output of audio data. The input / output I / F 26 includes a USB connector and the like, is connected to the imaging unit 14, and controls acquisition of captured images from the imaging unit 14. Further, the input / output I / F 26 includes a VGA terminal, an IEEE1394 connector, and the like, and can be connected to an input device such as a keyboard and a mouse, a display device such as a liquid crystal display, and the like to control these devices. Here, the imaging unit 14 is connected to the input / output I / F 26, but may be connected to the communication I / F 24. Further, the projecting unit 12 may be connected to the input / output I / F 26 instead of the communication I / F 24.

  In FIG. 2, the PC is exemplified as a configuration that does not include the above input device or the like, but the PC may be configured to include an input device or the like. In addition, the PC is not limited to the HDD 23, and an SSD (Solid State Drive) may be used. The communication I / F 24 is connected to the projection unit by either a wired connection or a wireless connection depending on a connection form with the projection unit. Communication may be performed.

  FIG. 3 is a diagram exemplifying that the image of a user at a remote place is projected on the screen 11 by the remote communication system shown in FIG. In FIG. 3, a portion indicated by a black circle is the imaging means 14 arranged at a predetermined position on the screen 11. Actually, the lens of the imaging means 14 is disposed in a window provided on the screen 11, and the imaging element is disposed behind the lens.

  As shown in FIG. 3, the image of the user is displayed so that the imaging unit 14 is arranged at the middle position between both eyes of the user at a remote place. You can communicate with your eyes. Here, an example is shown in which an image is displayed so that the imaging unit 14 is arranged at an intermediate position between both eyes of a user at a remote location, but the position of the imaging unit 14 is not limited to the intermediate position, and the remote Any position may be used as long as the position is close to both eyes of the local user. This is because it is not possible to recognize that the other party is looking at the video and the line of sight is deviated if it is in such a nearby position.

  FIG. 4 is a diagram exemplifying that a material is projected on the screen 11 together with a video of a user at a remote place. The image display apparatus can project the received video and simultaneously project created or stored data. When projecting two images, a video and a document, the size of the video shown in FIG. 3 is reduced and the projection position is changed. In the example shown in FIG. 4, the screen 11 is divided into four equal parts, the reduced image is arranged in the upper left area, the material is arranged in the lower right area, and they are combined and projected. This is just an example, and you may project the two of them arranged side by side, you may project the top and bottom, or you may project a composite that partly overlaps Good. Note that the image data of the image to be projected may be downloaded from a server or the like via the network 10.

  Here, if the user who uses the remote communication system is an occupational physician and an employee, from the perspective of the occupational physician, if communication can be achieved through voice exchange, even if the employee's line of sight does not match, he / she does not care much . On the other hand, it is important for the counseling employees to ask the occupational physician to listen to the details of their consultation, and whether or not the occupational physician's perspective is directed at them. The feeling of security obtained is different. Usually, the consulting employee displays only the image of the occupational physician and has a conversation with the eyes of the occupational physician displayed.

  When the industrial physician is looking at the screen 11 shown in FIG. 3, since the projected image is only the image of the employee, it is possible to communicate by looking at the employee's eyes and communicating. As shown in FIG. 4, when two images are projected, the position of the imaging means 14 does not change, but the size and position of the employee's video are changed. For this reason, since the eyes of the industrial physician are not in line with the imaging means 14 when talking with the eyes of the employee, the employee sees the video where the industrial physician is looking at the upper left rather than himself. become.

  In addition, when the industrial doctor looks at the material projected at the same time, since the occupational physician's line of sight does not match the imaging means 14, the employee sees an image of the industrial physician looking at the lower right. This makes it impossible to communicate with the eyes.

  In view of this, an image display device installed on the side of the industrial physician is provided with a function for controlling the position of the image and the material to be projected in accordance with the eye of the industrial physician. That is, in the above example, the image projected on the upper left area is moved to a position where the imaging means 14 is located, or the material projected on the lower right area is moved to a position where the imaging means 14 is located. . More specifically, the image or document so that the position viewed by the industrial physician in the video and the position viewed by the industrial physician in the document coincide with the position indicated by the black circle with the imaging means 14. Move. As a result, since the occupational physician sees the imaging means 14 and has a conversation, it is possible to communicate by matching the eyes. Hereinafter, although the user who uses an image display apparatus is demonstrated as an industrial physician and an employee, it is not limited to this. Note that the image display device installed on the employee side may or may not have the function of controlling the position of the above-described video and materials.

  One example of the functional configuration of the image display apparatus installed on the occupational physician side is shown in FIG. In FIG. 5, an apparatus used by an industrial physician is an image display apparatus 30, and an apparatus used by an employee is an image display apparatus 40. Data exchanged between the image display device 30 and the image display device 40 includes video data and audio data, which are one of the image data. Here, description will be made assuming that only video data is exchanged. The image display device 30 includes a position control device 31 and a projection unit 12 in order to realize the function. The projection means 12 receives the video data acquired from the image display device 40, projects it on the screen 11, and displays the employee's video. The position control device 31 includes an imaging unit 14 and a communication unit 33. The imaging means 14 images an industrial physician and outputs a video of the industrial physician as video data. The communication unit 33 transmits the video data output from the imaging unit 14 to the image display device 40 and receives the video data transmitted from the image display device 40.

  The image display device 30 further includes a detection unit 32, an arrangement instruction unit 34, a first reproduction unit 35, a second reproduction unit 36, and a synthesis unit 37. The detecting means 32 detects the direction of the line of sight of the industrial physician, calculates the position viewed by the industrial physician on the screen 11 from the direction, and notifies the arrangement instruction means 34 of the positional information. The detection unit 32 can detect the direction of the line of sight of the industrial physician based on the image of the industrial physician imaged by the imaging unit 14. For example, it is possible to detect the direction of the line of sight from the position of the iris with respect to the reference point, with the reference point as the head or the corner of the eye. For this purpose, the position of the iris with respect to the reference point and the direction of the line of sight are associated with each other and accumulated in a database, and the accumulated data can be used for the determination.

  The detection means 32 obtains the position on the screen 11 that the industrial physician is looking at based on the detected line-of-sight direction, the distance between the screen 11 and the industrial physician, and the size of the screen 11. The position can be obtained as two-dimensional coordinates (x, y) on the screen 11 plane. The coordinate unit may be a pixel, or a predetermined rectangular area (block) including a certain number of pixels vertically and horizontally.

  For example, for five points at the four corners and the center of the screen 11, the relationship between the coordinates of the five points and the direction of the line of sight is obtained in advance based on the resolution and the direction when the points are viewed. The relationship is held as a table, a relational expression, or the like, and the position on the screen 11 viewed by the industrial physician can be obtained from the detected line-of-sight direction using the relationship.

  The arrangement instruction unit 34 uses the position information notified from the detection unit 32 and the position information of the imaging unit 14 to determine whether or not the line of sight is shifted. A determination means may be provided separately for this determination. Here, it is determined whether or not it is necessary to move the content such as the projected image or document and change its arrangement. The deviation of the line of sight can be determined based on whether or not the position on the screen 11 viewed by the industrial physician and the position on the screen 11 of the imaging means 14 are the same. Further, the line-of-sight shift can be determined based on whether or not the position on the screen 11 viewed by the industrial physician is within a predetermined range including the position of the imaging means 14. Note that the predetermined range can be determined as an allowable range in which it is possible to determine that the line of sight of an industrial physician is looking at the user when the screen is viewed as an employee.

  When it is determined that the line of sight is deviated as a determination result, the arrangement instruction unit 34 calculates how much and in what direction the content at the position viewed by the industrial physician should be moved. The arrangement instructing unit 34 instructs the first reproducing unit 35 and the second reproducing unit 36 to change the arrangement including the direction and distance as deviation information. The first reproducing unit 35 requests the synthesizing unit 37 to project the video data received by the communication unit 33. The first reproduction means 35 decodes the video data as necessary. In response to the instruction from the arrangement instruction unit 34, the first reproduction unit 35 changes the size and position of the projected image. The second reproducing unit 36 requests the synthesizing unit 37 to project the material created by the image display device 30 or the stored material. In response to the instruction from the arrangement instruction unit 34, the second reproduction unit 36 also changes the size and position of the displayed material. The position is changed based on the deviation information included in the above instruction.

  When receiving a request from the first reproducing unit 35 or a request from the second reproducing unit 36, the synthesizing unit 37 projects the projection on the projecting unit 12 without changing the size or position of the image or the material. Request. When the request is received from both the first playback unit 35 and the second playback unit 36, the combining unit 37 arranges the video and the material at the instructed size and position, combines them, and the combined image Projection is requested to the projection means 12. The projecting unit 12 projects the requested composite image on the screen 11.

  In addition to the above method, the direction of the line of sight can also be detected from, for example, corneal reflection and pupil position of an image obtained by irradiating an industrial physician with infrared rays from an infrared LED and photographing with an infrared camera. Specifically, the direction of the line of sight can be detected from the distance between the corneal reflection and the pupil and the direction of the pupil with respect to the corneal reflection. Also in this method, the position from the reference point and the information on the direction of the line of sight can be stored in the database in association with each other, and the stored data can be used.

  For this reason, as shown in FIG. 6, it can be set as the structure further equipped with the infrared irradiation means 38 which irradiates infrared rays, and the infrared imaging means 39 which images the to-be-photographed object irradiated with infrared rays. The infrared irradiation means 38 can be the above infrared LED, and the infrared imaging means 39 can be the above infrared camera. So far, two methods for detecting the direction of the line of sight have been illustrated, but the present invention is not limited to these two methods, and any method known so far can be used.

  5 and 6, the image display device 30 includes the projection unit 12. However, the display of the video is not limited to the projection on the screen 11, and the display unit 30 may display the image on the screen. Good. Therefore, the image display apparatus 30 can also display the composite image by providing the projection unit 12 without requesting the external display unit by the synthesis unit 37. Further, the image display device 30 may be provided with a display unit and displayed on the screen.

  The processing executed by the image display device 30 shown in FIG. 5 will be described in detail with reference to the sequence diagrams shown in FIGS. In the image display device 30, in order to display the created material or the stored material, the second reproduction unit 36 requests the composition unit 37 to perform projection as shown in FIG. 7. For example, the second reproduction unit 36 updates the drawing of the material or the like using an application such as a text editor or paint, and requests the synthesis unit 37 to project the updated image. At that time, when the instruction from the arrangement instructing unit 34 is received in advance, the second reproducing unit 36 performs drawing according to the instruction, designates the display position, and projects the image on the synthesizing unit 37. Request. On the other hand, when the instruction from the arrangement instructing unit 34 is not received, the second reproducing unit 36 designates the display position according to the value set as the set condition, and the image is sent to the synthesizing unit 37. Request projection of

  The synthesizing unit 37 synthesizes the image requested from the second reproducing unit 36 and any other requested image, and requests the projecting unit 12 to project the synthesized image. To do. The projecting unit 12 projects the requested composite image on the screen 11.

  Next, processing performed when an image is acquired by the imaging unit 14 will be described with reference to FIG. The imaging unit 14 can acquire, for example, 30 images at 30 fps, that is, 30 images per second. The imaging unit 14 transfers the acquired image to the communication destination image display device 40 via the communication unit 33. The image display device 40 acquires the image and projects it on a screen using a projection unit.

  The imaging unit 14 sends the acquired image to the detection unit 32 in parallel therewith. The detecting unit 32 determines whether or not there is a person in the image captured by the image capturing unit 14, and when there is a person, detects the direction of the line of sight of the person. Based on the detected direction of the line of sight, the detection unit 32 obtains the position on the screen 11 that the person is looking at and notifies the arrangement instruction unit 34 of the position information.

  The arrangement instructing unit 34 determines a line-of-sight shift from the position of the image capturing unit 14 and the notified position information. When it is determined that a shift has occurred, the arrangement instructing unit 34 notifies the first reproduction unit 35 and the second reproduction unit 36. On the other hand, it is instructed which position to move to and what size to change. When receiving the instruction, the first reproduction unit 35 and the second reproduction unit 36 retain the instruction content and change the display content based on the instruction content at the next display update time.

  Next, processing performed when video data is transmitted from the image display device 40 will be described with reference to FIG. The communication unit 33 receives video data from the image display device 40 and sends the video data to the first reproduction unit 35. The first reproduction means 35 decodes the video data as necessary and decomposes it into images for each frame. The first reproducing unit 35 requests the synthesizing unit 37 to project the decomposed image in accordance with the instruction from the arrangement instructing unit 34. The synthesizing unit 37 that has received this request arranges the image received from the first reproducing unit 35 and the requested image, if there is another requested image, based on the instructed position, synthesizes, and projects The means 12 is requested to project the synthesized image. The projecting unit 12 accepts the request and projects the synthesized image onto the screen 11.

  The synthesizing unit 37 requests the projecting unit 12 to project only the image requested by the first reproducing unit 35 when no other requested image exists. At this time, the synthesizing unit 37 requests the projecting unit 12 to project according to the set condition. Specifically, the projection of the image is requested so that the position of the imaging unit 14 is the middle position between both eyes of the person in the image. Then, the projection unit 12 projects only the image requested by the first reproduction unit 35 onto the screen 11. The image on the screen 11 in this case is as shown in FIG.

  An example of an image projected on the screen 11 by such processing is shown in FIGS. 10 and 11 show images when the images requested by the first reproduction unit 35 and the second reproduction unit 36 are synthesized by the synthesis unit 37 and the synthesized image is projected by the projection unit 12. FIG. FIGS. 10 and 11 are diagrams when at least one of the two images shown in FIG. 4 is moved so that communication can be performed with the eyes aligned.

  In FIG. 4, when an industrial physician is looking at a portion indicated by a gray circle, when a remote employee looks at the industrial physician displayed on the screen, it appears to be facing the lower right. For this reason, the image display apparatus 30 on the occupational physician side moves the projected synthesized image so as to correct the shift between the position of the imaging means 14 and the position of the portion viewed by the occupational physician. As described above, the movement of the image is obtained by determining to which position the image projected by the arrangement instructing unit 34 is moved and changing to such a size, instructing it, and performing the first reproducing unit 35 and the second reproducing unit. This is done when the means 36 requests the combining means 37 in accordance with the instruction.

  In the example shown in FIG. 10, the arrangement instructing unit 34 instructs both the first reproducing unit 35 and the second reproducing unit 36 to move, and the two images are moved in the upper left direction without changing the size. ing. That is, all the displayed contents are moved. Here, the position of the line of sight 15 of the portion viewed by the industrial physician is moved so as to coincide with the position of the imaging means 14 indicated by a black circle. For this reason, most of the employee's image is off the screen 11 and projected outside the screen 11.

  In this way, the two images can be moved together, but one image can be fixed at that position and only the other image can be moved. An example is shown in FIG. In FIG. 11, the arrangement instructing unit 34 does not instruct the first reproducing unit 35 to change the position, but only instructs the second reproducing unit 36 to change the position. In other words, the arrangement instructing unit 34 makes the occupational physician with respect to the second reproducing unit 36 so that the position of the line of sight 15 of the portion viewed by the industrial physician matches the position of the imaging unit 14 indicated by a black circle. Instruct to change the position of the image being viewed.

  In the example shown in FIGS. 10 and 11, only the position of the image is changed, but the size of the image can be changed to make it easy to see the contents. In that case, by changing the size, the position of the line of sight of the part viewed by the industrial physician can be enlarged around the position so that it does not move. Further, after enlargement or the like, the position may be moved so as to coincide with the position of the imaging means 14.

  So far, an example has been shown in which the occupational physician's viewing position does not change, that is, the occupational physician is constantly looking at one place. However, in reality, the line of sight moves from time to time to the part of interest. For example, as shown in FIG. 12, there are many cases in which the user moves from the line of sight 15 indicated by a gray circle to the line of sight 16 and returns to the line of sight 15 again. If the image is moved in such a way as to follow the movement of the line of sight, the display becomes unsettled. If the frequency of the movement is high, it becomes impossible to match the line of sight to what you want to see.

  Therefore, the time (stay time) that is stopped at the destination position after the line of sight moves is measured, and if the stay time exceeds a certain time (threshold), it is determined that there is a shift in the line of sight. , It can be controlled to move the image at that position. When the frame rate of a moving image is, for example, 30 fps and the threshold is 1 second, the position viewed by an industrial physician obtained from one frame worth of images (30 images) is not changed after the line of sight moves In addition, the position can be moved from the next projected image. The threshold value is not limited to 1 second, and may be 0.5 seconds, 2 seconds, or the like. Although an example of counting by the number of frames is shown here, it is also possible to measure time using a timer or the like that measures time.

  A person's line of sight does not continue to look at a single point, but basically moves vertically and horizontally. Accordingly, the line of sight of the industrial physician is constantly moving around the line of sight 15 to line of sight 17, as indicated by the gray circles in FIG. For example, when the line of sight moves largely and then repeats small movements at the destination, it is assumed that the staying time of the line of sight is less than or equal to a threshold value, the image is not moved, Deviation may occur in the position. This makes it impossible to communicate with the eyes.

  On the other hand, when the line of sight moves within a narrow range, an employee watching an industrial doctor's image does not recognize that the line of sight has shifted to this extent. Therefore, when the line of sight remains within the predetermined range 50 that is not recognized as the line of sight being shifted, it can be determined that there is no change in the position viewed by the industrial physician. As a result, even when the line of sight moves once and repeats small movements at the destination, the image is appropriately moved to the destination, and the actual viewing position and the position of the imaging means 14 Can be eliminated.

  In this case, there are a plurality of line-of-sight positions within the range 50, and it becomes a problem which line-of-sight position is used to move to match the position of the imaging means 14. However, since any of the positions is within a range where it is not recognized that the line of sight is shifted, any position may be used as a reference. Therefore, it may be the line-of-sight position that first enters the range 50, or may be the line-of-sight position last in the range 50. It is also possible to average the coordinate values of the x coordinate and the coordinate value of the y coordinate of a plurality of line-of-sight positions, and use the average coordinate value obtained for each. The range 50 may be each area when the screen 11 is equally divided into a predetermined number in advance, or may be a rectangular area corresponding to a predetermined pixel vertically and horizontally with the first line-of-sight position moved as a center. In this case, the image is moved so that the position of the imaging means 14 and the line-of-sight position do not coincide with each other but approach within a predetermined distance.

  As described above, by detecting the direction of the line of sight, obtaining the viewing position on the screen 11, and moving the viewed image in accordance with the deviation of the position from the position of the imaging means 14, It is possible to communicate with other users by matching their eyes. As a result, the user can consult with peace of mind in places such as health consultation and mental health counseling.

  The position control device, image display device, and image display method of the present invention have been described in detail with reference to the embodiments shown in the drawings. The present invention is not limited to the above-described embodiments, and other embodiments, additions, changes, deletions, and the like can be modified within a range that can be conceived by those skilled in the art. As long as the functions and effects of the present invention are exhibited, the scope of the present invention is also included. Therefore, it is possible to provide a program for causing a computer to execute the image display method, a recording medium on which the program is recorded, and the like.

DESCRIPTION OF SYMBOLS 10 ... Network, 11, 11a, 11b ... Screen, 12a, 12b ... Projection means, 13a, 13b ... Information processing means, 14, 14a, 14b ... Imaging means, 15-17 ... Line of sight, 20 ... CPU, 21 ... ROM, 22 ... RAM, 23 ... HDD, 24 ... Communication I / F, 25 ... External storage I / F, 26 ... Input / output I / F, 27 ... Audio input device, 28 ... Audio output device, 29 ... Bus, 30, 40 ... Image display device, 31 ... Position control device, 32 ... Detection means, 33 ... Communication means, 34 ... Arrangement instruction means, 35 ... First reproduction means, 36 ... Second reproduction means, 37 ... Synthesis means, 38 ... Infrared irradiation Means 39 ... Infrared imaging means 50 ... Range

Japanese Patent No. 3011439

Claims (20)

A position control device for controlling the position of content displayed on a display surface of a display means,
An imaging means that is arranged at a predetermined position on the display surface and images a user facing the display surface;
Reproducing means for displaying a plurality of contents on the display surface according to a set condition or instruction;
Detection means for detecting a position on the display surface that the user is viewing based on the captured image;
Determination means for determining a shift of the line of sight of the user with respect to the position of the imaging means based on the position detected by the detection means;
At least of the plurality of contents so that the position of the imaging unit and the position on the display surface viewed by the user match or approach within a predetermined distance according to a determination result by the determination unit A position instructing unit that instructs the reproduction unit to change the arrangement of the content at the position viewed by the user on the display surface;   The position control apparatus according to claim 1, wherein the arrangement instruction unit instructs the reproduction unit to change all arrangements of the plurality of contents displayed on the display surface.   The said arrangement | positioning instruction | indication means instruct | indicates the change of arrangement | positioning only of the content in the position which the said user is viewing among the said several content displayed on the said display surface with respect to the said reproduction | regeneration means. Position control device.   In response to the determination that the displacement from the determination unit is present, the arrangement instruction unit is at least at a position viewed by the user based on the position detected by the detection unit and the position of the imaging unit. The position control device according to any one of claims 1 to 3, wherein a direction and a distance in which a certain content is moved are calculated, and the calculated direction and the distance are provided as deviation information to the reproduction unit.   5. The position according to claim 4, wherein the determination unit determines that the line of sight of the user is deviated when the position detected by the detection unit is not within a predetermined range including the position of the imaging unit. Control device.   6. The determination unit according to claim 5, wherein the determination unit determines that the user's line of sight has a deviation when the position detected by the detection unit is not within the predetermined range even after a predetermined time has elapsed. Position control device.   The arrangement instruction means averages the coordinate values of the plurality of positions detected by the detection means within the predetermined time, and uses the obtained average coordinate values to calculate the direction and the distance. Item 7. The position control device according to Item 6. Infrared irradiation means for irradiating the user with infrared rays, and infrared imaging means for imaging the user irradiated with the infrared rays,
The position control device according to claim 1, wherein the detection unit detects a position on the display surface viewed by the user based on an image captured by the infrared imaging unit. The display means includes a screen having the display surface, and projection means for projecting the plurality of contents on the screen.
The position control device according to claim 1, wherein the imaging unit is disposed on a back surface of the screen and images the user through a window provided at a predetermined position of the screen. A program for causing a computer to execute a process for controlling the position of content displayed on a display surface of a display means.
Displaying a plurality of contents on the display surface of the display means according to a set condition;
Imaging a user facing the display surface by an imaging means arranged at a predetermined position on the display surface;
Detecting a position on the display surface that the user is viewing based on the captured image;
Determining a shift of the line of sight of the user with respect to the position of the imaging means based on the detected position;
Of the plurality of contents, the position of the imaging unit and the position on the display surface that the user is viewing match or approach within a predetermined distance according to the determination result in the determination step. Instructing to change the arrangement on the display surface of the content at least at the position viewed by the user;
A program that receives an instruction to change the arrangement, and executes a step of changing the arrangement of at least a position of the plurality of contents displayed on the display surface that the user is viewing.   The program according to claim 10, wherein in the instructing step, an instruction to change all arrangements of the plurality of contents displayed on the display surface is given.   The program according to claim 10, wherein in the instructing step, an instruction to change the arrangement of only the content at the position viewed by the user among the plurality of contents displayed on the display surface is given.   In the instructing step, upon receiving the determination that the deviation is present in the determining step, the content at least at the position viewed by the user is moved based on the detected position and the position of the imaging means. The program according to any one of claims 10 to 12, wherein a direction and a distance to be calculated are calculated.   The program according to claim 13, wherein in the determining step, when the detected position is not within a predetermined range including the position of the imaging unit, it is determined that there is a deviation in the line of sight of the user.   The program according to claim 14, wherein in the determination step, it is determined that there is a deviation in the line of sight of the user when the detected position is not within the predetermined range even after a predetermined time has elapsed.   16. The instructing step averages the coordinate values of a plurality of the positions detected within the predetermined time, and uses the obtained average coordinate values to calculate the direction and the distance. Program. Further executing a step of irradiating the user with infrared rays by infrared irradiation means and a step of imaging the user irradiated with the infrared rays by infrared imaging means;
The program according to any one of claims 10 to 16, wherein, in the detecting step, a position on the display surface that the user is viewing is detected based on an image captured by the infrared imaging unit. The display means includes a screen having the display surface, and projection means for projecting the plurality of contents on the screen.
The step of imaging by the imaging unit includes imaging the user through a window provided at a predetermined position of the screen by the imaging unit arranged on the back surface of the screen. The listed program. An image display device that displays a plurality of contents,
A screen having a display surface;
Projecting means for projecting the plurality of contents onto the screen;
An imaging means that is arranged at a predetermined position on the back surface of the screen, and that images a user facing the display surface through a window provided on the screen;
Playback means for requesting the projection means to project the plurality of contents on the display surface according to a set condition or instruction;
Detection means for detecting a position on the display surface that the user is viewing based on the captured image;
Determination means for determining a shift of the line of sight of the user with respect to the position of the imaging means based on the position detected by the detection means;
At least of the plurality of contents so that the position of the imaging unit and the position on the display surface viewed by the user match or approach within a predetermined distance according to a determination result by the determination unit An image display apparatus, comprising: an instruction unit for instructing the reproduction unit to change the arrangement of the content at the position viewed by the user on the display surface. An image display method for displaying a plurality of contents,
Projecting the plurality of contents onto a screen having a display surface;
Imaging a user facing the display surface through a window provided in the screen by imaging means arranged at a predetermined position on the back of the screen;
Detecting a position on the display surface that the user is viewing based on the captured image;
Determining a shift of the line of sight of the user with respect to the position of the imaging means based on the detected position;
Of the plurality of contents, the position of the imaging unit and the position on the display surface that the user is viewing match or approach within a predetermined distance according to the determination result in the determination step. Instructing to change the arrangement on the display surface of the content at least at the position viewed by the user;
Receiving an instruction to change the arrangement, and changing the arrangement of content at a position viewed by at least the user among the plurality of contents displayed on the display surface.