JP2016021206A - Display device, display system, display method and display program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow an image to be displayed at a proper position for a plurality of persons.SOLUTION: A display device has a position detection unit that detects the position of a person in a predetermined range including a display area, a distance calculation unit that calculates distance among a plurality of persons if the plurality of persons were detected, and an output control unit that allows one or more image(s) to be displayed according to the distance in the display area.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a display device, a display system, a display method, and a display program that display an image in a display area.

  2. Description of the Related Art Conventionally, display devices used for displaying materials in a presentation during a meeting, displaying various contents in a store, and the like are known. In addition, with conventional display devices, it is known that, as the screen size increases, an image obtained by reducing the displayed screen is displayed on the screen, and an operation on the displayed screen is performed via this image. Yes.

  For example, Patent Literature 1 discloses a technique in which a window for controlling an application program is displayed in the vicinity of a plurality of users in a video display unit, and the display position of the window follows the user's movement. In this technique, when the distance between users becomes less than the appropriate distance, the windows for each user are displayed so as not to overlap.

  In the conventional technology described above, it is difficult to display an image (window) at an appropriate position for each person, for example, when a plurality of persons are arranged side by side.

  In the disclosed technology, a position detection unit that detects the position of a person within a predetermined range including a display area, and a distance calculation unit that calculates a distance between a plurality of persons when there are a plurality of detected persons, An output control unit configured to display one or a plurality of images according to the distance in the display area.

  The disclosed technology can display an image at an appropriate position for a plurality of persons.

It is a figure explaining the outline | summary of 1st embodiment. It is a figure which shows an example of the hardware constitutions of the display apparatus of 1st embodiment. It is a figure explaining the functional structure of the display apparatus of 1st embodiment. It is a figure explaining integration and composition of images. It is a sequence diagram explaining operation | movement of the display apparatus of 1st embodiment. It is a figure explaining calculation of the distance between persons by a distance calculation part. It is a figure which shows an example of the screen which sets a 1st and 2nd threshold value. It is a flowchart explaining the process of a determination part and an image generation part. It is a figure explaining expansion of the integrated image. It is a figure explaining the adjustment of the display area of the integrated image. It is a figure explaining the image which performs the operation instruction of a screen. It is a figure explaining the case where an integrated screen is not separated. It is a figure explaining integration in case any one of the screens before integration receives the operation instruction. It is a 1st figure explaining separation when the screen after integration receives the operation instruction. It is a 2nd figure explaining separation when the screen after integration receives the operation instruction. It is a figure explaining the functional structure of the display apparatus of 2nd embodiment. It is a figure which shows an example of correspondence information. It is a flowchart explaining the process of a person analysis part, a corresponding | compatible image selection part, a determination part, and an image generation part. It is a figure which shows the 1st example of a display system. It is a figure which shows the 2nd example of a display system. It is a figure which shows the 3rd example of a display system. It is a figure which shows an example of the hardware constitutions of a server.

(First embodiment)
The outline of the first embodiment will be described below with reference to the drawings.

  In the present embodiment, when a plurality of persons are detected in the captured image when the screen displayed by the display device is captured, an image corresponding to each person is displayed in the vicinity of the person on the screen.

  Further, in the present embodiment, when a plurality of persons come close to each other and are adjacent to each other in the captured image, two images corresponding to each person are integrated or combined and displayed. Furthermore, in this embodiment, when adjacent users are separated from each other in the captured image, the integrated or combined image is separated and returned to the image for each person before integration. Details of integration and synthesis in this embodiment will be described later.

  FIG. 1 is a diagram for explaining the outline of the first embodiment. In the present embodiment, the screen 10 displayed by the display device is a large screen used for, for example, a presentation. The large screen in the present embodiment is a screen having such a size that there is a range that cannot be reached when a person standing in front of the screen tries to give an operation instruction on the screen.

  FIG. 1 illustrates an example of a captured image when the screen 10 is captured by the imaging device. FIG. 1A shows an example of a state in which a plurality of persons A and B in front of the screen 10 are detected in the captured image 1A including the screen 10. FIG. 1B shows a state in which the persons A and B are close to each other and the distance between the two persons is less than a predetermined first threshold in the captured image 1B. FIG. 1C illustrates a state in which the persons A and B are separated from each other and the distance between the persons is equal to or greater than a predetermined second threshold in the captured image 1C. Details of the first threshold and the second threshold will be described later.

  In FIG. 1A, an image 11 corresponding to the person A and an image 12 corresponding to the person B are displayed in the vicinity of the persons A and B, respectively. The vicinity mentioned here may be, for example, a position overlapping with a person on the screen 10 in the captured image. More specifically, for example, the position on the screen 10 may overlap with the person's head, or the position that can be indicated by the person's arm.

  In the present embodiment, when the distance between the persons A and B decreases from the state of FIG. 1A, the images 11 and 12 move in the screen 10 following the movements of the persons A and B, respectively. In the present embodiment, as shown in FIG. 1B, when the distance between the persons A and B is less than the first threshold, the images 11 and 12 corresponding to the persons are displayed as one image 13. Integrated or synthesized.

  In the following description of the embodiment, a state in which the distance between a plurality of persons is less than a first threshold is referred to as a state in which the persons are adjacent to each other.

  In the present embodiment, as shown in FIG. 1C, when the persons A and B are separated and the distance between the persons A and B is equal to or greater than the second threshold, the image 13 is changed to the persons A and B. The corresponding images 11 and 12 are separated, and the images 11 and 12 are made to follow the movements of the persons A and B in the screen 10. That is, in the present embodiment, when a plurality of persons are detected within a predetermined range including the image display area, the plurality of persons are handled at positions corresponding to the distances between the plurality of persons in the display area. Display the selected image. The image display area is the screen 10, and the predetermined range is an imaged range.

  In the present embodiment, when there are a plurality of persons in front of the screen 10 by the above processing, an image can be displayed at an appropriate position for the plurality of persons.

  Below, the display apparatus of this embodiment is demonstrated. FIG. 2 is a diagram illustrating an example of a hardware configuration of the display device according to the first embodiment.

  The display device 100 according to the present embodiment includes an input device 101, an output device 102, an auxiliary storage device 103, a memory device 104, an arithmetic processing device 105, an interface device 106, and an imaging device 107, which are mutually connected by a bus B. .

  The input device 101 is used for inputting various signals to the display device 100. Specifically, the input device 101 may be, for example, an operation member that receives an operation on the display device 100. The output device 102 is used to output an image to be displayed, and is a device that displays image data. Specifically, for example, when the display device 100 is a projector or the like, the output device 102 is a projection mechanism or the like for projecting an image. For example, when the display device 100 is an information processing device having a display, the output device 102 is a display.

  For example, the auxiliary storage device 103 may store a display program installed in the display device 100. Further, the auxiliary storage device 103 may store image data of an image output from the output device 102, for example.

  The memory device 104 is used for temporary storage or the like when the arithmetic processing device 105 or the like performs processing. The arithmetic processing unit 105 realizes various processes as described later according to the display program.

  The interface device 106 includes a modem, a LAN card, etc., and is used for connecting to a network. The imaging device 107 images a predetermined range including the screen displayed by the output device 102. Specifically, the imaging apparatus 107 according to the present embodiment is a camera that captures a still image or a moving image, for example.

  The display program of this embodiment is at least a part of various programs that control the display device 100. The display program is provided, for example, by distributing a recording medium or downloading from a network. The recording medium on which the display program is recorded is a recording medium that records information optically, electrically, or magnetically, such as a CD-ROM, a flexible disk, or a magneto-optical disk; Various types of recording media, such as a semiconductor memory for recording data, can be used.

  The display program downloaded from the network is installed in the auxiliary storage device 103 via the interface device 106.

  The display device 100 of the present embodiment may be a projector that displays an image by projecting an image, for example. In addition, the display device of the present embodiment may be an information processing device having a large display or the like as the output device 102.

  Next, functions of the display device 100 of the present embodiment will be described with reference to FIG. FIG. 3 is a diagram illustrating a functional configuration of the display device according to the first embodiment.

  The display device 100 according to the present embodiment includes a coordinate storage unit 210 and an image storage unit 220. The coordinate storage unit 210 stores coordinate information indicating the coordinates of a person detected from an image captured by the imaging device 107. That is, the coordinate storage unit 210 of this embodiment is a collection of coordinate information.

  The image storage unit 220 of the present embodiment stores image data of an image output from the output device 102. That is, the image storage unit 220 of the present embodiment is a collection of image data.

  In addition, the display device 100 according to the present embodiment has the display program 200 installed therein, and the arithmetic processing device 105 executes the display program 200, thereby realizing the functions of the units described below.

  The display device 100 according to the present embodiment includes a position detection unit 230, a coordinate management unit 240, a distance calculation unit 250, a determination unit 260, an image generation unit 270, an output control unit 280, and an imaging control unit 290.

  The position detection unit 230 detects the presence of a person from the captured image captured by the imaging device 107. Specifically, the position detection unit 230 detects a captured image or a person using, for example, face authentication technology.

  In addition, when a person is detected in the captured image, the position detection unit 230 generates person identification information (hereinafter referred to as a person ID) and assigns the person ID to the coordinate information indicating the position of the person in the captured image. For example, when a plurality of persons are detected in the captured image, the position detection unit 230 according to the present embodiment generates a person ID for each detected person and assigns the person ID to the coordinate information for each person. Therefore, in this embodiment, when a plurality of persons are detected in the captured image, coordinate information is managed for each person.

  The coordinate management unit 240 stores the coordinate information assigned with the person ID in the coordinate storage unit 210. The coordinate management unit 240 also reads out the person ID and coordinate information from the coordinate storage unit 210.

  The distance calculation unit 250 acquires coordinate information of each person when a plurality of persons are detected, and calculates a distance between the persons from the coordinate information. Details of the calculation of the distance between persons will be described later.

  Based on the distance between the persons calculated by the distance calculation section 250 and the first threshold value, the determination section 260 determines whether to integrate or combine images corresponding to each person. In addition, the determination unit 260 determines whether or not to separate the combined or combined images based on the distance between persons and the second threshold value.

  The image generation unit 270 generates image data output from the output device 102 and stores it in the image storage unit 220. More specifically, the image generation unit 270 displays the image data of the image displayed on the screen 10, the image data of the image 11 corresponding to the person A, the image data of the image 12 corresponding to the person B, and the image 11. , 12 are integrated or synthesized (see FIG. 1).

  The output control unit 280 controls the output device 102 to display the image data stored in the image storage unit 220.

  The imaging control unit 290 controls imaging by the imaging device 107. Specifically, the imaging control unit 290 controls the imaging device 107 so that an image including the screen 10 is captured at predetermined intervals.

  Next, with reference to FIG. 4, “integration” and “synthesis” of images in the image generation unit 270 of this embodiment will be described. FIG. 4 is a diagram for explaining integration and synthesis of images.

  FIG. 4A is a diagram illustrating image integration. In the present embodiment, combining the two images 11A and 12A into one area is referred to as “integration” of images.

  In the display device 100, the case where the images are integrated is a case where the image 11A and the image 12A displayed on the screen 10 are reduced images of the image displayed on the screen 10, for example. That is, the case where the images are integrated is a case where the image data that is the source when the image generation unit 270 generates the image 11A and the image data that is the source when the image 12A is generated are the same.

  The case where the image data that is the basis of the image 11A and the image 12A is, for example, a case where the image 11A and the image 12A are images of an operation screen for performing an operation on the screen 10 by reducing the screen 10. . In this case, the image data that is the basis when the image generation unit 270 generates the image 11 </ b> A and the image 12 </ b> A is both image data of the image displayed in the screen 10.

  In the display device 100 of the present embodiment, when the distance between the person A and the person B becomes less than the first threshold, the image generation unit 270 generates an integrated image 13A in which the image 11A and the image 12A are integrated and outputs the integrated image 13A. It is displayed in the screen 10 by the control unit 280.

  FIG. 4B is a diagram for explaining image composition. In the present embodiment, displaying at least a part of the image 11B and a part of the image 12B in one area is referred to as “synthesis” of the images. In the present embodiment, for example, the entire images 11B and 12B may be displayed in one area.

  In the display device 100, the case where the images are combined is, for example, the case where the image 11B displayed on the screen 10 is an image corresponding to the person A and the image 12B is an image corresponding to the person B. That is, the case where the images are combined is a case where the image data that is the source when the image generation unit 270 generates the image 11B is different from the image data that is the source when the image 12B is generated.

  The case where the image data from which the image 11B and the image 12B are based is different may be, for example, the case where the image 11B and the image 12B are advertisement images corresponding to the person A and the person B, respectively.

  In the display device 100 of the present embodiment, when the distance between the person A and the person B is less than the second threshold value, the composite image 13B obtained by combining the image 11B and the image 12B is displayed on the screen 10.

  In the following description of the present embodiment, it is assumed that images are integrated when the distance between a plurality of persons becomes less than a second threshold.

  With reference to FIG. 5, the operation of the display device 100 of the present embodiment will be described. FIG. 5 is a sequence diagram for explaining the operation of the display device according to the first embodiment. FIG. 5 is a sequence diagram when a plurality of persons are detected in the captured image.

  When a person detection request is input, the display device 100 according to the present embodiment acquires a captured image captured by the imaging device 107, detects a person by the position detection unit 230, and outputs the detection result to a coordinate management unit. 240 is notified (step S501). More specifically, the position detection unit 230 generates a person ID for each detected person, and notifies the coordinate management unit 240 of the generated person ID and coordinate information indicating the position of the person in the captured image.

  Upon receiving the notification of the person ID and the coordinate information, the coordinate management unit 240 stores the person ID and the coordinate information in the coordinate storage unit 210, and updates the coordinate storage unit 210 (step S502). Subsequently, the coordinate management unit 240 passes the person ID and the coordinate information to the image generation unit 270, and notifies the image generation unit 270 of an instruction to generate image data of an image to be displayed at a position based on the coordinate information (step S503). ).

  Upon receiving this notification, the image generation unit 270 requests the determination unit 260 to determine whether to perform image integration or image separation (step S504). In response to this request, the determination unit 260 notifies the distance calculation unit 250 of a request for calculating a distance between persons (step S505). Note that the determination unit 260 of the present embodiment notifies the distance calculation unit 250 of the person ID generated in step S501 together with the distance calculation request.

  When receiving the distance calculation request and the person ID, the distance calculation unit 250 acquires coordinate information corresponding to the received person ID from the coordinate storage unit 210 (step S506). Subsequently, the distance calculation unit 250 calculates the distances between the plurality of persons detected in step S501 from the acquired coordinate information (step S507).

  Subsequently, the distance calculation unit 250 notifies the determination unit 260 of the calculated distance (step S508). The determination unit 260 determines an image to be generated based on the calculated distance between persons and the first and second threshold values, and notifies the image generation unit 270 of the result (step S509). In response to the notification from the determination unit 260, the image generation unit 270 acquires image data from the image storage unit 220 (step S510). Then, the image generation unit 270 generates image data of an image to be displayed on the screen 10 (step S511). Note that the image generated in step S511 is an image corresponding to each detected person. Subsequently, the image generation unit 270 stores the generated image data in the image storage unit 220 (step S512). The image generation unit 270 may store the image data with the person ID generated in step S501 in the image storage unit 220.

  Subsequently, the image generation unit 270 makes a display request for image data to the output control unit 280 (step S513). When receiving the display request, the output control unit 280 acquires the image data generated by the image generation unit 270 in step S511 from the image storage unit 220 (step S514). Note that when the image generation unit 270 notifies the output control unit 280 of a display request, the image generation unit 270 may also notify the person ID assigned to the image data. The output control unit 280 may acquire the corresponding image data from the image storage unit 220 based on the person ID notified together with the display request. Subsequently, the output control unit 280 displays the acquired image data on the screen 10 (step S515), and ends the process.

  Details of the processing from step S506 to step S512 by the distance calculation unit 250, the determination unit 260, and the image generation unit 270 will be described later.

  Next, calculation of the distance between persons by the distance calculation unit 250 of the present embodiment will be described with reference to FIG. FIG. 6 is a diagram for explaining the calculation of the distance between persons by the distance calculation unit.

  In FIG. 6, the distance calculation unit 250 calculates the distance L between the person A and the person B detected in the captured image 60.

  The distance calculation unit 250 of the present embodiment calculates the distance between the persons A and B for each captured image from the positions of the persons A and B in the past several captured images captured by the imaging device 107 at predetermined intervals, for example. The distance L may be calculated from the shortest distance.

  In the example of FIG. 6, the coordinate information of each of the persons A and B is acquired in the past three captured images. In FIG. 6, rectangular areas indicating the positions of the persons A and B in the first captured image are denoted as A1 and B1, respectively. Similarly, in FIG. 6, the rectangular areas indicating the positions of the persons A and B in the second captured image are A2 and B2, respectively, and the rectangular areas indicating the positions of the persons A and B in the third captured image are A3 and B3, respectively. It was. The rectangular area of the present embodiment is indicated by the coordinates of four points that form a rectangle in the captured image 60.

  In the example of FIG. 6, the distance between the rectangular area A3 and the rectangular area B2 is the shortest. Therefore, the distance calculation unit 250 may set the distance L between the side 61 in the rectangular area A3 and the side 62 closest to the side 61 in the rectangular area B2.

  As described above, in the present embodiment, the distance calculation accuracy can be improved by calculating the distance between the persons A and B using a plurality of captured images. The distance calculation unit 250 may calculate the distance between the persons A and B from one captured image, for example.

  Next, the first threshold value and the second threshold value will be described with reference to FIG. In the present embodiment, the determination unit 260 integrates the images 11 and 12 corresponding to the persons A and B when the distance calculated by the distance calculation unit 250 is less than the first threshold. In the present embodiment, the first and second threshold values may be set in advance in the input device 101 of the display device 100 or the like.

  FIG. 7 is a diagram illustrating an example of a screen for setting the first and second threshold values.

  The screen 70 illustrated in FIG. 7 includes an input field 71 for inputting a first threshold value for determining integration and a second threshold value for determining separation.

  In addition to the first and second threshold value input fields, the screen 70 has selection fields 72, 73, and 74 for inputting qualitative settings for integration and separation. In the display device 100 of the present embodiment, first and second threshold values corresponding to the selection fields 72 to 74 may be set in advance.

  For example, when the selection field 73 is selected on the screen 70, the determination unit 260 may set first and second threshold values corresponding to “near”. Similarly, for example, when the selection column 74 is selected, the determination unit 260 may set the first and second threshold values corresponding to “distant”.

  In the present embodiment, the first and second threshold values corresponding to the selection fields 72 to 74 may be changed according to the size of the display area of the images 11 and 12 corresponding to the person.

  For example, when the display areas of the images 11 and 12 corresponding to the persons A and B are wide, the first and second threshold values when the selection field 73 is selected are the display values of the images 11 and 12 compared to this case. The value may be smaller than the first and second threshold values when the region is narrow.

  Furthermore, in the present embodiment, the first and second threshold values corresponding to the selection columns 72 to 74 may be set according to the detected height of the person. In this case, the display device 100 preferably stores the height of the person and the first and second threshold values corresponding to the selection fields 72 to 74 in association with each other.

  In the present embodiment, the first threshold value and the second threshold value may be the same value or different values.

  Next, details of the processing from step S506 to step S512 by the determination unit 260 and the image generation unit 270 will be described with reference to FIG. FIG. 8 is a flowchart for explaining processing of the determination unit and the image generation unit.

  In the display device 100 of the present embodiment, the position detection unit 230 detects a person in the captured image (step S801) and determines whether there are a plurality of detected persons (step S802). If there are a plurality of detected persons in step S802, the distance calculation unit 250 calculates a distance between the plurality of persons (step S803). If it is determined in step S802 that there are not a plurality of persons, the process proceeds to step S810 described later.

  Subsequently, the determination unit 260 determines whether images corresponding to a plurality of persons have already been integrated (step S804). If the images are integrated in step S804, the determination unit 260 determines whether the distance between persons calculated in step S803 is equal to or greater than a second threshold (step S805).

  In step S805, when the distance between persons is equal to or greater than the second threshold, the image generation unit 270 generates image data obtained by separating the integrated image (integrated image) into a plurality of images before integration. The generated image data is stored in the image storage unit 220 (step S806). Subsequently, the display device 100 causes the output control unit 280 to display an image before integration in the vicinity of each person based on the image data stored in the image storage unit 220 in step S806 (step S807), and ends the process. .

  If the distance between persons is not equal to or greater than the second threshold value in step S805, the output control unit 280 displays the image data of the integrated image stored in the image storage unit 220 (step S808), and ends the process. When displaying the integrated image, the output control unit 280 of the present embodiment displays the integrated image at a position based on the positional relationship between a plurality of persons. Specifically, for example, the output control unit 280 may display the intermediate point of the distance between persons so as to be the center of the integrated image.

  If the images are not integrated in step S804, the determination unit 260 determines whether the distance between the persons is less than the first threshold (step S809). If it is less than the first threshold value in step S809, the image generation unit 270 generates image data obtained by integrating images corresponding to each of a plurality of persons, and stores it in the image storage unit 220 (step S810). Subsequently, the display device 100 causes the output control unit 280 to display the generated image data (step S811) and ends the process.

  In step S809, when the distance between persons is not less than the first threshold, the image generation unit 270 causes the output control unit 280 to display an image corresponding to each person stored in the image storage unit 220 (step S812). The process is terminated.

  In the display device 100 according to the present embodiment, when there are not a plurality of persons detected in step S802, that is, when the number of detected persons is one, the image generation unit 270 corresponds to the person from the image storage unit 220. An image to be acquired is acquired and displayed (step S813), and the process ends.

  In the present embodiment, as described above, when a plurality of persons are detected in the captured image, the images corresponding to the persons are integrated (synthesized) and separated according to the distance between the persons. Therefore, in this embodiment, for example, when a plurality of persons are adjacent to each other, an image can be displayed at an appropriate position for the plurality of persons.

  In the present embodiment, when the images corresponding to the persons A and B are integrated by the image generation unit 270, the integrated images may be enlarged. FIG. 9 is a diagram for explaining enlargement of an integrated image.

  In FIG. 9, an image 91 is an image corresponding to the person A, and an image 92 is an image corresponding to the person B.

  In FIG. 9, since images 91 and 92 each correspond to one person, the size of the display area of the images 91 and 92 is determined when one person views or operates the image. It is a suitable size.

  On the other hand, an image 93 obtained by integrating the image 91 and the image 92 is an image that two persons A and B view and operate. Therefore, the image generation unit 270 according to the present embodiment generates image data of the image 93 so that the display area is larger than the display areas of the images 91 and 91.

  In this embodiment, the display area is enlarged by enlarging the image after integration in this way, thereby improving convenience when a plurality of persons browse or operate the image after integration.

  In the present embodiment, the size of the display area of the image 93 after integration may be adjusted according to the distance between the person A and the person B after the images 91 and 91 are integrated.

  FIG. 10 is a diagram illustrating adjustment of the display area of the integrated image.

  In the present embodiment, the image generation unit 270 adjusts the size of the display area of the image 93 according to the distance L between the person A and the person B after generating the image data of the image 93 described in FIG. You may do it.

  For example, when the distance L becomes smaller than the first predetermined value after the image 93 is displayed, the image generation unit 270 generates and displays the image data of the image 94 in which the display area of the image 93 is narrowed. May be. When the distance L becomes larger than the second predetermined value after the image generation unit 270 displays the image 93, the image generation unit 270 displays the image data of the image 95 in which the display area of the image 93 is widened. May be generated and displayed.

  In the present embodiment, after the images 91 and 92 are integrated as described above, the display area of the image 93 is adjusted according to the distance L, so that the size can be easily browsed or operated based on the standing positions of the persons A and B. An integrated image can be displayed. Therefore, in this embodiment, an image can be displayed at an appropriate position for each of the plurality of persons in a state where the images corresponding to the plurality of persons are integrated.

  Here, the relationship between the first and second predetermined values and the first and second threshold values is defined as first predetermined value <second predetermined value <first and second threshold values. In the example of FIG. 10, two predetermined values are used for adjusting the display area of the image 93, but the present invention is not limited to this. The display area of the integrated image may be adjusted stepwise according to the distance L, for example.

  Next, with reference to FIG. 11, a case will be described in which the display device 100 of the present embodiment is a projector, and images corresponding to a plurality of persons displayed on the screen are screen images for instructing operation of the screen. To do.

  FIG. 11 is a diagram for explaining an image for instructing operation of a screen. A display device 100 shown in FIG. 11 is a projector. A screen 110 illustrated in FIG. 11 is a presentation screen projected by the display device 100.

  The example in FIG. 11 assumes a scene in which two people A and B make a presentation. That is, in the example of FIG. 11, it is assumed that the person A performs an operation on the screen 110 via the screen 111 and the person B performs an operation on the screen 110 via the screen 112.

  The screens 111 and 112 are reduced screens of the screen 110, are displayed on the screen 110, and accept operation instructions for the screen 110 from the persons A and B, respectively. That is, screens 111 and 112 in FIG. 11 are the same screen. The display device 100 in FIG. 11 changes the image to be displayed on the screen 110 in accordance with the operation instruction received on the screens 111 and 112.

  Note that the display device 100 according to the present embodiment may, for example, analyze the movements of the persons A and B in the captured image, and transition the image on the screen 110 based on the type of operation corresponding to the movements of the persons A and B. .

  More specifically, the display device 100 may include, for example, an operation table in which a person's action is associated with an operation instruction. In this case, when the actions of the persons A and B are, for example, actions that slide the screen 110 in the horizontal direction, the display device 100 refers to the operation table and displays the screen based on the corresponding operation instruction. The display of 110 may be controlled.

  In FIG. 11, the screen 110 is a screen projected by a projector, but the present invention is not limited to this. The screen 110 may be displayed on, for example, a television. When the screen 110 is a display such as a television, for example, and the screen 110 has an operation reception function, the operation instructions received on the screens 111 and 112 may be transmitted to the display device 100 main body. The transmission of the operation instruction from the display to the display device 100 may be performed, for example, via a wired cable connecting the display and the display device 100, a wireless LAN (Local Area Network), or the like.

  When the distance L between the person A and the person B becomes less than the first threshold, the image generation unit 270 generates and displays a screen in which the screens 111 and 112 are integrated.

  Next, with reference to FIGS. 12 to 15, screen separation or integration while the screens 111, 112, or the screen 121 are accepting operation instructions will be described.

  The display device 100 according to the present embodiment does not separate the screen 121 when the distance L between the persons A and B is equal to or larger than the first threshold while the screen 121 is accepting the operation instruction. Hereinafter, a case where the integrated screen 121 is not separated will be described.

  FIG. 12 is a diagram illustrating a case where the integrated screen is not separated.

  In the display device 100 according to the present embodiment, the distance L between the person A and the person B becomes equal to or larger than the first threshold while the screen 121 in which the screens 111 and 112 are integrated is accepting the operation instruction. In this case, the screen 121 is not separated. The display device 100 according to the present embodiment may display a message 122 on the screen 121 indicating that the screen 121 is being operated.

  Specifically, in the analysis of the captured image including the screen 121, the display device 100 has the distance L between the persons A and B equal to or greater than the first threshold and the screen 121 is accepting an operation instruction. In this case, the image generation unit 270 displays the message 122 on the screen 121. In addition, the display device 100 according to the present embodiment includes, for example, a speaker (not shown), and may notify the persons A and B that the screen 121 is being operated by voice together with the display of the message 122. . Further, the display device 100 may notify the persons A and B that the screen 121 is being operated only by voice.

  When the distance L between the persons A and B is less than the first threshold while either the screen 111 or the screen 112 is accepting an operation instruction, the display device 100 according to the present embodiment displays the screen 111. , 112, the brightness of the screen that has not received an operation instruction is reduced.

  FIG. 13 is a diagram for explaining integration when any one of the screens before integration accepts an operation instruction.

  In FIG. 13, the screen 111 is in the process of accepting an operation instruction by the person A, and the screen 112 is in a state of not accepting the operation instruction. In this state, when the distance L between the persons A and B is less than the first threshold, the image generation unit 270 reduces the brightness of the screen 112 that has not received an operation instruction, and accepts the operation instruction for the screen 112. To stop.

  Specifically, the display device 100 may analyze only the operation of the person A corresponding to the screen 111 in the captured image including the screens 111 and 112 and accept it as an operation instruction.

  In the example of FIG. 13, the image generation unit 270 may halve the luminance of the screen 112, for example. In FIG. 13, by reducing the brightness of the screen 112, the person B corresponding to the screen 112 can recognize that it is impossible to accept an operation instruction for the screen 112. In the present embodiment, a message indicating that the operation instruction cannot be accepted may be displayed on the screen 112.

  In the example of FIG. 13, when the distance L is less than the first threshold, the image generation unit 270 arranges the screen 111 and the screen 112 with reduced brightness until the reception of the operation instruction for the screen 111 is completed. It may be displayed. Then, the image generation unit 270 may integrate the screens 111 and 112 after the reception of the operation instruction is completed.

  In this way, the person A can continue to give operation instructions to the screen 111.

  Next, with reference to FIGS. 14 and 15, the case where the distance L is equal to or greater than the second threshold while the integrated screen 121 is accepting an operation instruction will be described.

  FIG. 14 is a first diagram illustrating separation when the screen after integration accepts an operation instruction.

  The display device 100 separates the screen 121 when the person B moves away from the person A and the distance L is equal to or greater than the second threshold while the screen 121 is accepting an operation instruction from the person A. The brightness of the screen 112 is reduced, and operation instructions for the screen 112 are not accepted. Further, the display device 100 continuously accepts operation instructions for the screen 111 separated from the screen 121 from when the screen 121 is displayed.

  In the present embodiment, by separating the screen 121 as described above, the operation instruction of the person A who has performed the operation on the screen 121 can be continuously received.

  FIG. 15 is a second diagram for explaining separation when the integrated screen accepts an operation instruction.

  In the example of FIG. 15, when the distance L is equal to or greater than the second threshold while the integrated screen 121 is accepting the operation instruction, the operation instruction accepted by the screen 121 is discarded, and the fact is displayed. After being displayed on the screen 121, the screen 121 is separated into the screen 111 and the screen 112.

  In the example of FIG. 15, the person A issues an operation instruction 123 on the screen 121. In this state, when the distance L becomes equal to or greater than the second threshold, the image generation unit 270 displays a message 124 on the screen 121 indicating that the screen 121 is to be separated and that the operation instruction is invalid.

  Then, the image generation unit 270 separates the screen 121 into the screen 111 and the screen 112. At this time, the operation instruction 123 by the person A is invalidated on the screen 111, and the screen 111 is displayed as the screen 111 before the operation instruction 123 is received.

  Therefore, in the example of FIG. 15, after the operation instruction performed on the screen 121 after integration is discarded, it can be performed again on the screen 111 after separation.

  As described above, in the present embodiment, when a plurality of persons are detected in a captured image obtained by capturing a large screen displayed by the display device 100, image integration is performed according to the distance between the plurality of persons. And separation. Therefore, according to the present embodiment, it is possible to display an image at an appropriate position for a plurality of persons.

  This embodiment can be understood as shown below. That is, the display device 100 according to the present embodiment includes a position detection unit 230 that detects the position of a person in the captured images 1A, 1B, 1C, and 60 including the screen 10, and a plurality of detected persons. A distance calculation unit 250 that calculates a distance L between the two persons, and output control for displaying images 11, 12, 13, 91, 92, and 93 corresponding to a plurality of persons at positions corresponding to the distance L on the screen 10. Part 280.

  The present embodiment also includes an image generation unit 270 that generates image data to be displayed by the output control unit 280. The image generation unit 270 includes a plurality of persons when the distance L is less than the first threshold. Image data of an image obtained by integrating a plurality of images corresponding to the respective images is generated.

  In the present embodiment, the image generation unit 270 corresponds to each of a plurality of persons when the distance L is equal to or greater than the second threshold after generating the image data in which the images are integrated. The image data is separated into a plurality of images.

  In this embodiment, the image generation unit 270 generates image data of the integrated image having a size corresponding to the distance L when the distance L is less than the first threshold.

  In the present embodiment, images corresponding to a plurality of persons are images 111 and 112 that accept screen operation instructions displayed on the screen 110, and the distance is displayed while the operation instructions are accepted in the integrated image. When L is equal to or greater than the second threshold, the output control unit 280 outputs a message 122 that does not separate the integrated image data.

  Further, in the present embodiment, the distance L is less than the first threshold while receiving an operation instruction in at least one of the plurality of images 111 and 112 corresponding to each of a plurality of persons. At this time, the image generation unit 270 reduces the luminance of an image other than the image that is being accepted.

  The display method of the present embodiment is a display method executed by the display device 100, and is a position detection procedure for detecting the position of a person in the captured images 1A, 1B, 1C, and 60 including the screen 10. When there are a plurality of detected persons, in step S507, which is a distance calculation procedure for calculating the distance L between the plurality of persons, and an image corresponding to the plurality of persons at positions corresponding to the distance L on the screen 10. Step S515, which is an output control procedure for displaying.

  The display program 200 according to the present embodiment includes a position detection step S501 for detecting the positions of persons in the captured images 1A, 1B, 1C, and 60 including the screen 10, and a plurality of persons when the detected persons are plural. The display device 100 is caused to execute a distance calculation step S507 for calculating a distance L between the two and an output control step S515 for displaying images corresponding to a plurality of persons at positions corresponding to the distance L on the screen 10.

(Second embodiment)
The second embodiment will be described below with reference to the drawings. In the second embodiment, the image corresponding to the person is, for example, various contents such as an advertisement. In the following description of the second embodiment, those having the same functional configuration as in the first embodiment are given the same reference numerals as those used in the description of the first embodiment, and the description thereof is omitted. To do.

  FIG. 16 is a diagram illustrating a functional configuration of the display device according to the second embodiment.

  In the display device 100A of the present embodiment, information indicating the characteristics of a plurality of persons detected in the captured image is acquired, and an image associated with the information indicating the characteristics is displayed as an image corresponding to each person. In the following description, information indicating the characteristics of a person is called feature information.

  A display program 200A is installed in the display device 100A of the present embodiment. The display device 100A according to the present embodiment includes a correspondence information storage unit 225, a person analysis unit 295, and a correspondence image selection unit 298. The correspondence information storage unit 225 is a collection of information indicating the correspondence between the feature information and the image to be displayed. In the following description, information indicating the correspondence between the feature information and the image to be displayed is referred to as correspondence information. Details of the correspondence information will be described later.

  The person analysis unit 295 analyzes an image of a person in the captured image and acquires feature information of the person shown in the captured image.

  The corresponding image selection unit 298 refers to the correspondence information storage unit 225 and selects an image corresponding to the feature information.

  FIG. 17 is a diagram illustrating an example of correspondence information. The correspondence information stored in the correspondence information storage unit 225 of the present embodiment includes feature information and image type as information items, and the two are associated with each other. The feature information shown in FIG. 17 includes, for example, the gender of a person and the height of the person.

  In FIG. 17, for example, when the gender of a person is male, “tie”, “suit”, and the like are associated as image types, and when the gender is female, the type of image is “cosmetics”. "Is associated.

  Moreover, when the height of a person is less than 100 cm, it is assumed that the person is a child. Therefore, in the example of FIG. 17, when the height is less than 100 cm as the person characteristic information, “toy” or the like is associated as the image type.

  For example, when a person is detected in the captured image, the corresponding image selection unit 298 according to the present embodiment refers to the correspondence information storage unit 225 and corresponds to the feature information in the image data stored in the image storage unit 220. Image data of a type of image may be selected and passed to the image generation unit 270.

  Next, operations of the person analysis unit 295, the corresponding image selection unit 298, the determination unit 260, and the image generation unit 270 in the display device 100A of the present embodiment will be described with reference to FIG.

  FIG. 18 is a flowchart for explaining processing of the person analysis unit, the corresponding image selection unit, the determination unit, and the image generation unit.

  In the display device 100A of the present embodiment, the position detection unit 230 detects a person in the captured image (step S1801). When a person is detected in the captured image, the person analysis unit 295 analyzes the image of the person included in the captured image and acquires feature information (step S1802). The feature information may be held together with the person ID and the person coordinate information.

  The processing from step S1803 to step S1806 is the same as the processing from step S802 to step S805 in FIG. 8 except that the images corresponding to a plurality of persons are combined, and thus the description thereof is omitted.

  If the distance L is greater than or equal to the second threshold value in step S1806, the image generation unit 270 separates the synthesized image into images corresponding to the feature information of each person (step S1807). The display device 100A causes the output control unit 280 to display the separated image data (step S1808), and ends the process.

  In step S1806, when the distance L is less than the second threshold value, the display device 100A displays the combined image (step S1809) and ends the process. The synthesized image is an image obtained by synthesizing images corresponding to the feature information of a plurality of persons.

  The processing in step S1810 is the same as the processing in step S809 in FIG.

  In step S1810, when the distance L is less than the first threshold, the display device 100A combines the image corresponding to the feature information of each person by the image generation unit 270 (step S1811), and the output control unit 280. The synthesized image is displayed (step S1812), and the process ends.

In step S1810, when the distance L is equal to or greater than the first threshold, the display device 100A acquires the correspondence information storage unit 225 by using the correspondence image selection unit 298, selects an image corresponding to the feature information of each person, and outputs it. The image selected by the control unit 280 is displayed for each person (step S1813).

  If the detected person is not plural in step S1803, the corresponding image selection unit 298 selects an image corresponding to the detected feature information of the person, and displays the image selected by the output control unit 280 ( Step S1814).

  As described above, in the present embodiment, an image corresponding to the feature information of the person detected in the captured image is displayed. Therefore, in the present embodiment, for example, an image in which a detected person is more likely to be interested can be displayed. For example, in displaying an advertisement image, the advertising effect can be enhanced.

  This embodiment can be understood as shown below. That is, the display device 100A of the present embodiment includes a position detection unit 230 that detects the position of a person in the captured images 1A, 1B, 1C, and 60 including the screen 10, and a plurality of detected persons when there are a plurality of detected persons. A distance calculation unit 250 that calculates a distance L between the persons, and an output control unit 280 that displays images corresponding to a plurality of persons at positions corresponding to the distance L on the screen 10, and the captured image 1A 1B, 1C, and 60 are captured images captured by the imaging device 107, and a person analysis unit 295 that acquires the feature information of the person detected by analyzing the captured images 1A, 1B, 1C, and 60, A correspondence image selection unit 298 that refers to correspondence information in which feature information and image types are associated, and selects images corresponding to the feature information as images 11, 12, and 13 corresponding to a plurality of persons. You .

(Third embodiment)
The third embodiment will be described below with reference to the drawings. The third embodiment is different from the first and second embodiments in that a part of the configuration of the display device of the first and second embodiments is provided in the external device. In the following description of the third embodiment, the same reference numerals as those used in the first and second descriptions are given to those having the same functional configuration as those in the first and second embodiments. Description is omitted.

  FIG. 19 is a diagram illustrating a first example of the display system. A display system 300 according to the present embodiment includes a display device 100B and a server 400 connected via a network.

  In the display system 300, the display program 200 is installed in the display device 100B, and the coordinate storage unit 210 and the image storage unit 220 are not included. In the present embodiment, the coordinate storage unit 210 and the image storage unit 220 are provided in the server 400. In the present embodiment, the storage area used in the display device 100B can be reduced by holding the coordinate storage unit 210 and the image storage unit 220 in the external server 400 in this way, and the processing load of the display device 100B is reduced. Can be reduced.

  In the display system 300, the coordinate storage unit 210 and the image storage unit 220 are provided in the server 400, but the present invention is not limited to this. The coordinate storage unit 210 and the image storage unit 220 may be provided in, for example, a storage area provided outside the display device 100B. For example, the coordinate storage unit 210 and the image storage unit 220 are provided not in the server 400 but in a USB memory that can be attached to and detached from the display device 100B. Also good.

  FIG. 20 is a diagram illustrating a second example of the display system. The display system 300A of this embodiment includes a display device 100C and a server 400A connected via a network.

  In the example of FIG. 20, the server 400A includes a coordinate storage unit 210, an image storage unit 220, a position detection unit 230, a coordinate management unit 240, a distance calculation unit 250, a determination unit 260, and an image generation unit 270. The display device 100C includes an output control unit 280 and an imaging control unit 290.

  That is, in the display system 300A, the display device 100C only controls the output device 102 and the imaging device 107, and other functions are realized by the server 400A. Accordingly, the display device 100C illustrated in FIG. 20 is a display device that transmits the captured image captured by the imaging device 107 to the server 400A and outputs the image data generated by the image generation unit 270 of the server 400A by the output device 102.

  FIG. 21 is a diagram illustrating a third example of the display system. The display system 300B according to the present embodiment includes a display device 100D, a server 400A, and an imaging device 107A connected via a network. In the display system 300B of FIG. 21, a captured image is acquired using the imaging device 107A provided outside the display device 100D. The display device 100D may or may not include the imaging device 107.

  In this case, the server 400A acquires a captured image from the imaging device 107A, and transmits the image data generated by the image generation unit 270 based on the captured image to the display device 100D. The display device 100D outputs the image data received from the server 400A by the output device 102.

  In the example of FIG. 21, the imaging device 107A is connected to the server 400A and the display device 100D via a network, but the present invention is not limited to this. The imaging device 107A may be directly connected to the display device 100D.

  Note that the examples shown in FIGS. 19 to 21 can be similarly applied to the second embodiment. Further, the servers 400 and 400A of the present embodiment may be realized by a general information processing apparatus having an arithmetic processing device and a storage device.

  Hereinafter, the hardware configuration of the servers 400 and 400A of the present embodiment will be described. FIG. 22 is a diagram illustrating an example of a hardware configuration of the server. Since the hardware configurations of the server 400 and the server 400A are the same, the server 400 will be described with reference to FIG.

  The server 400 includes an input device 41, an output device 42, a drive device 43, an auxiliary storage device 44, a memory device 45, an arithmetic processing device 46, and an interface device 47 that are mutually connected by a bus B.

  The input device 41 includes a keyboard and a mouse, and is used for inputting various signals. The output device 42 includes a display device and the like, and is used for displaying various windows and data. The interface device 47 includes a modem, a LAN card, etc., and is used for connecting to a network.

  A part of the display program 200 installed in the server 400 is at least a part of various programs that control the server 400. A part of the display program 200 is provided, for example, by distributing the recording medium 48 or downloading from the network. A recording medium 48 on which a part of the display program is recorded is a recording medium such as a CD-ROM, a flexible disk, a magneto-optical disk, etc. for recording information optically, electrically or magnetically, a ROM, a flash memory, etc. Various types of recording media such as a semiconductor memory that electrically records information can be used.

  A part of the display program 200 is installed from the recording medium 48 to the auxiliary storage device 44 via the drive device 43 when the recording medium 48 on which the display program is recorded is set in the drive device 43. A part of the display program 200 downloaded from the network is installed in the auxiliary storage device 44 via the interface device 47.

  The auxiliary storage device 44 stores a part of the installed display program 200 and also stores necessary files, data, and the like. The memory device 45 reads and stores a part of the display program from the auxiliary storage device 44 when the computer is activated. Then, the arithmetic processing unit 46 implements the various processes described above according to a part of the display program stored in the memory device 45.

  As described above, in this embodiment, by providing a part of the functions realized by the display programs 200 and 200A in an external device such as a server, the processing load on the display device can be reduced.

  This embodiment can be understood as shown below. That is, the display systems 300 and 300A of the present embodiment are the display systems 300 and 300A including the display devices 100B to 100D and the servers 400 and 400A connected to the display devices 100B to 100D. A position detection unit 230 that detects the positions of persons in the images 1A, 1B, 1C, and 60; a distance calculation unit 250 that calculates a distance L between the plurality of persons when there are a plurality of detected persons; The screen 10 includes an output control unit 280 that displays images 11, 12, 13, 91, 92, 93 corresponding to a plurality of persons at positions corresponding to the distance L.

100, 100A to 100D Display device 200, 200A Display program 210 Coordinate storage unit 220 Image storage unit 225 Corresponding information storage unit 230 Position detection unit 240 Coordinate management unit 250 Distance calculation unit
260 determination unit 270 image generation unit 300 display system

Japanese Patent No. 5260643

Claims (11)

A position detector that detects the position of a person within a predetermined range including a display area;
A distance calculating unit that calculates a distance between a plurality of persons when there are a plurality of detected persons;
A display device comprising: an output control unit configured to display one or a plurality of images according to the distance in the display area.   A determination unit that determines to display one image when the distance is less than a first threshold, and determines to display a plurality of images when the distance is less than a second threshold; The display device according to claim 1. An image generation unit that generates image data of an image to be displayed by the output control unit;
The image generation unit
When the distance is less than the first threshold, generate image data of one image obtained by integrating a plurality of images corresponding to each of the plurality of persons,
The output control unit
The display device according to claim 2, wherein the one image is displayed at a position based on a positional relationship between the plurality of persons in the display area. The image generation unit
When the distance is equal to or greater than the second threshold after generating image data in which images are integrated, the integrated image data is separated into image data of a plurality of images corresponding to each of the plurality of persons. ,
The output control unit
The display device according to claim 3, wherein the plurality of images are displayed at positions corresponding to positions of the plurality of persons in the display area. The image generation unit
In the case where the distance is less than the first threshold,
The display device according to claim 3, wherein image data of the integrated image having a size corresponding to the distance is generated. The images corresponding to the plurality of persons are
An image for accepting an operation instruction for a screen displayed in the display area;
In the middle of accepting the operation instruction in the integrated image, when the distance becomes a second threshold or more,
The display device according to claim 3, wherein the output control unit outputs a notification that does not separate the integrated image data. When the distance is less than the first threshold while receiving the operation instruction in at least one of the plurality of images corresponding to each of the plurality of persons,
The image generation unit
The display device according to claim 6, wherein the brightness of an image other than the image that is being accepted is reduced. The predetermined range is a captured image captured by an imaging device,
A person analysis unit that obtains characteristic information of the person detected by analyzing the captured image;
2. A correspondence image selection unit that refers to correspondence information in which the feature information is associated with an image type, and selects an image corresponding to the feature information as an image corresponding to the plurality of persons. The display apparatus as described in any one of thru | or 7. A display system including a display device and an information processing device connected to the display device,
A position detector that detects the position of a person within a predetermined range including a display area;
A distance calculating unit that calculates a distance between a plurality of persons when there are a plurality of detected persons;
A display system including: an output control unit configured to display one or a plurality of images according to the distance in the display area; A display method executed by a computer,
A position detection procedure for detecting the position of a person within a predetermined range including a display area;
A distance calculation procedure for calculating a distance between a plurality of persons when there are a plurality of detected persons;
An output control procedure for displaying one or a plurality of images according to the distance in the display area. A position detecting step for detecting the position of the person within a predetermined range including the display area;
A distance calculating step for calculating a distance between the plurality of persons when there are a plurality of detected persons;
A display program for causing a computer to execute an output control step of displaying one or a plurality of images according to the distance in the display area.

Images