JP2013080394A - Display control device, control method of display control device, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To select information to be displayed according to an area of a detected object in a taken image and contents of attribute information, in an AR (Augmented Reality) system which displays the attribute information corresponding to the object in the taken image by making a composite image.SOLUTION: There is provided a display control device which comprises: identifier acquisition means for acquiring an identifier of another device; information acquisition means for acquiring information associated with the identifier; detection means for detecting an object corresponding to the identifier acquired by the identifier acquisition means from a taken image; and selection means for selecting information to be displayed according to the information, which is acquired by the information acquisition means and corresponds to the object detected by the detection means, and an area of the object in the taken image.

Description

  The present invention relates to a display control apparatus that displays information about an object in a captured image.

  In recent years, there is an augmented reality (AR) technology in which attribute information of an object in a captured image is added to and synthesized by a captured image captured by a camera or the like. For example, there is a technique for synthesizing and displaying attribute information of an object in a captured image based on position information obtained by GPS (Global Positioning System).

  Further, based on the feature information used for identification acquired from the mobile terminal of the subject person, the subject person in the captured image is specified, the attribute information of the specified person is acquired from the server, and the specified subject in the captured image is acquired. A technique for displaying information in the vicinity of the above has been proposed (Patent Document 1).

  In addition, there is a technique for giving a visual superiority or inferiority relationship to the display of graphic information displayed in association with each subject in accordance with distance information to the camera of each subject in the image. In Patent Document 2, for example, graphic information displayed in association with a certain person is displayed so as not to overlap with the other person when the other person is in front. On the other hand, the graphic information displayed in association with a certain person is displayed so as to overlap the graphic information displayed in association with the other person and the other person when the other person is behind.

JP 2002-305717 A JP2009-230431A

  In such an augmented reality system, it is desired to display attribute information related to the subject in association with the subject in the vicinity of the subject. That is, even when various subjects are included in the photographed image, it is desirable to display in a form that allows the user to know which subject information each attribute information related to the subject is. Therefore, it is necessary to control the display position of the attribute information in the image. FIG. 11 shows an example in which other subjects are hidden by displaying attribute information. As shown in FIG. 11, if the display position of the attribute information is not taken into consideration, there is a possibility that the subject displaying the attribute information cannot be identified.

  In addition, when there are many subjects for which attribute information is to be displayed in the image, even if the display position of the attribute information is displayed avoiding the face area of other subjects, all the attributes are displayed on the limited captured image. It is also possible that information cannot be displayed. Therefore, in view of whether there is a sufficient area for displaying the attribute information and the subject in association with each other, display control that preferentially displays attribute information useful for the user is desired.

  The above-described conventional patent document 1 does not consider the display position of attribute information and the selection of information to be displayed. In Patent Document 2, the superiority or inferiority relationship is determined according to the distance information to the camera of each subject in the image. However, when there are many subjects for which attribute information is to be displayed in the image, or the contents of the information It does not consider the superiority or inferiority of the display according to the.

  The present invention has been made in view of the above-described problems, and controls information displayed in a captured image in accordance with the area of the detected object in the captured image and the content of information related to the object. For the purpose.

  As means for solving the above-mentioned problems, there is a display control apparatus for displaying a captured image, an identifier acquisition means for acquiring an identifier of another apparatus, and an information acquisition means for acquiring information associated with the identifier Detection means for detecting an object corresponding to the identifier acquired by the identifier acquisition means from the captured image, information acquired by the information acquisition means corresponding to the object detected by the detection means, and detection by the detection means It is an object of the present invention to provide a display control apparatus comprising selection means for selecting the information to be displayed in accordance with an object area on the captured image.

  According to the present invention, information can be selected and displayed according to the information and the object area on the captured image. Therefore, for example, when all the attribute information and the corresponding object cannot be displayed in association with each other, it is possible to preferentially display the information considered to be desired by the user and the object corresponding to the information.

1 is a system configuration diagram according to the present embodiment. 2 is a module configuration diagram of the image display apparatus 101. FIG. 2 is a module configuration diagram of an information management server 102. FIG. It is a module block diagram of wireless tag 103a-c. It is a flowchart which shows an image display process. It is a flowchart which shows the process at the time of change of the classification of information. It is a figure which shows the sequence of the image display apparatus 101, the information management server 102, and wireless tag 103a-c. It is the figure which showed the mode at the time of the image display apparatus 101 displaying attribute information. It is the figure which showed an example of the mode when the image display apparatus 101 cannot display all the attribute information. It is the figure which showed the example in case attribute information overlaps with the attribute display apparatus 101. FIG. It is the figure which showed the mode of the display when the object corresponding to the other attribute information to display with the attribute information to display was hidden. It is an example of attribute information held by the wireless tags 103a to 103c. It is the figure which showed another example of the mode at the time of the image display apparatus 101 displaying attribute information. It is the figure which showed an example of the mode at the time of displaying the attribute information at the time of the change of the classification of information.

[Example 1]
Hereinafter, the present embodiment will be described in detail with reference to the drawings. FIG. 1 is a diagram illustrating a system configuration according to the present embodiment. Reference numeral 100 denotes a network. The network 100 according to the present embodiment may be various networks such as the Internet, a WAN (Wide Area Network), a LAN (Local Area Network), and a combination thereof. Reference numeral 101 denotes an image display device. The image display apparatus 101 acquires attribute information associated with an identifier uniquely identifying the terminal apparatus received from another terminal apparatus (wireless tag) from the information management server 102, and acquires the acquired attribute information, captured image, Is a display control device that synthesizes and displays them on the display unit. Further, the image display device 101 associates the attribute information of the object in the captured image with the object on the captured image and displays the combined information on the display unit. Accordingly, for example, information related to the owner of the terminal device (wireless tag) can be displayed superimposed on the display screen of the image display device 101, so that the user can easily acquire information related to the object of the captured image. Become. Further, since information is displayed in association with the object in the captured image, it is possible to easily identify which object emits what information.

  Reference numeral 102 denotes an information management server. When the information management server 102 receives an inquiry including the identification information of the terminal device (wireless tag) from the image display device 101 via the network 100, the information management server 102 displays the attribute information associated with the received identification information held by the image display device. 101. Further, the information management server 102 detects an object associated with the received identification information from the captured image, acquires feature information for specifying from the database, and transmits the feature information to the image display apparatus 101. In this embodiment, a reference image for performing object specifying processing by image processing is used as feature information. Note that the feature information is not limited to image data, and may be arbitrary feature information used for object specifying processing. In this embodiment, it is assumed that a person who is the owner of the wireless tag 103 is detected as an object. In addition, the information management server 102 associates attribute information actually displayed by the image display device with an identifier corresponding to the attribute information, and manages the display count based on transmission / reception of messages with the image display device.

  Reference numeral 103 denotes a wireless tag. The wireless tags 103a to 103c are held by subjects (people, buildings, etc.), and different identifiers are assigned to the wireless tags 103, respectively. The wireless tags 103a to 103c are terminal devices that transmit, as identification information, an identifier that can uniquely identify a terminal periodically or in response to a request from another terminal. The wireless tags 103a to 103c transmit identifiers in response to an identifier request from the image display apparatus 101 using a known wireless method (wireless LAN, millimeter wave, RFID, or the like).

  Next, the configuration of the image display apparatus 101 will be described with reference to FIG. The image display apparatus 101 has a CPU (Central Processing Unit). Each configuration to be described later is realized by the CPU executing a control program and controlling each hardware or calculating and processing information. Each configuration or part of the configuration may be configured by hardware. In FIG. 2, reference numeral 200 denotes a bus for connecting the modules. An imaging unit 201 includes a lens, an image sensor, and A / D conversion, captures subject light acquired through the lens, and generates the captured subject light as image data. A display unit 202 performs display control for displaying a captured image and various types of information on a display such as a liquid crystal panel. An identifier acquisition unit 203 acquires an identifier of the wireless tag 103 existing in a communicable range by controlling an RF unit 212 and a TCP / IP processing unit 211 described later. 204 controls the transmission / reception unit 206 (to be described later) to inquire the information management server 102 based on the identifier acquired by the identifier acquisition unit 203 and acquire the feature information associated with the identifier from the information management server 102. A feature information acquisition unit. 205 makes an inquiry to the information management server 102 based on the identifier acquired by the identifier acquisition unit 203 and acquires attribute information associated with the identifier from the server 111. The attribute information is information relating to the owner of the wireless tag in the present embodiment. For example, the attribute information includes the profile of the owner of the wireless tag, the object of interest, and the like. The attribute information may be information related to the corresponding object, and is not limited to the profile or the object of interest.

  Reference numeral 206 denotes a transmission / reception unit for transmitting / receiving a message to / from the information management server 102. Reference numeral 207 denotes a synthesis unit that creates synthesized image data by associating corresponding attribute information with an object in the vicinity of the image data and the object detected by the detection unit 210 on the image data. Reference numeral 209 denotes a determination unit that determines whether the acquired attribute information is information desired by the user and determines a priority indicating whether the attribute information should be displayed on each of the plurality of attribute information. Reference numeral 210 denotes hardware or a program for detecting a predetermined object from image data captured by the imaging unit 201 based on the feature information obtained by the feature information acquisition unit 204. In this embodiment, the detection unit 210 performs known face recognition processing, and detects an object corresponding to the feature information acquired by the feature information acquisition unit 204 from the captured image.

  A determination unit 208 determines whether to display each attribute information according to the priority determined by the acquired attribute information determination unit 209 and the area of each object on the captured image detected by the detection unit 210. It is. Further, the determination unit 208 determines a non-display area in which the attribute information in the captured image is not superimposed and displayed according to the area of each object in the image detected by the detection unit 210, and determines the display position of each attribute information. decide. A TCP / IP processing unit 211 performs communication protocol processing. Reference numeral 212 denotes an RF unit that controls an antenna and a circuit for transmitting and receiving a wireless signal to and from the wireless tag 103 by a predetermined wireless method (for example, wireless LAN, millimeter wave, RFID, etc.).

  Next, the functional configuration of the information management server 102 is shown in FIG. The information management server 102 has a CPU (Central Processing Unit). Each configuration to be described later is realized by the CPU executing a control program to control each hardware or perform calculation or processing of information. Each configuration or part of the configuration may be configured by hardware. In FIG. 3, reference numeral 300 denotes a bus for connecting the modules. A feature information holding unit 301 holds feature information for detecting and identifying an object associated with each identifier of the wireless tag 103 from the captured image. Reference numeral 302 denotes an attribute information holding unit that holds attribute information associated with each identifier of the wireless tag 103 attached to the subject. A transmission / reception unit 303 transmits / receives a message to / from the image display apparatus 101. In response to receiving a request including an apparatus identifier from the image display apparatus 101, the transmission / reception unit 303 transmits a message including attribute information and feature information corresponding to the identifier to the image display apparatus 101. A TCP / IP processing unit 304 performs communication protocol processing.

  Next, the module configuration of the wireless tag 103 will be described with reference to FIG. Reference numeral 401 denotes an RF unit that controls an antenna and a circuit for transmitting and receiving a wireless signal to and from the image display apparatus 101 using a predetermined wireless method (wireless LAN, millimeter wave, RFID, or the like). Reference numeral 402 denotes an identifier information transmission unit that controls the RF unit 401 to transmit a held identifier periodically or in response to a request from another device. The identifier is given and transmitted as an element of an Information Element of a beacon frame conforming to IEEE 802.11, for example.

  Next, the operation of the system having the above configuration will be described. First, processing at the time of image display in the image display apparatus 101 will be described with reference to the flowchart of FIG. When the image display device 101 starts the image display process, the identifier acquisition unit 203 transmits an identifier request message to the devices (wireless tags 103a to 103c) existing in a communicable range through the RF unit 212. Each wireless tag returns a message including its own tag ID (identifier) as a response to the request. The identifier acquisition unit 203 receives the response and acquires a tag ID (step S501). Subsequently, the feature information acquisition unit 204 transmits a feature information acquisition request to the information management server 102 including the acquired tag ID. The feature information acquisition unit 204 acquires feature information corresponding to the tag ID from the information management server 102. Similarly, the attribute information acquisition unit 205 acquires attribute information corresponding to the tag ID from the information management server 102 (step S502).

  In this embodiment, the feature information and the attribute information are acquired from the information management server 102. However, the present invention is not limited to this, and the information may be stored in the image display apparatus 101, and previously acquired information is reused. You may do it. In addition, the feature information acquisition unit 204 and the attribute information acquisition unit 205 of the image display apparatus 101 may acquire the feature information and attribute information directly from the wireless tag 103 without using the information management server 102. Next, the detection unit 210 performs analysis using the feature information acquired from the image captured by the imaging unit 201, and detects and identifies an object (subject) corresponding to the acquired identifier (step S503). In this embodiment, the analysis is performed in the image display apparatus 101. However, the present invention is not limited to this, and it can be realized by requesting an analysis from an external apparatus such as the information management server 102.

  The determination unit 209 determines whether the acquired attribute information is information desired by the user, and determines a priority indicating whether the attribute information should be displayed on each of the plurality of attribute information (step S504). In this embodiment, the image display apparatus 101 can set the type of information desired by the user. As information types, for example, it is assumed that there are a mode for displaying information about fashion, a mode for displaying information about food and drink, a mode for displaying information about hobbies, and the like. The system prepares a plurality of information type candidates in advance, and the user can select from the UI provided by the image display apparatus 101 the type related to the information he / she wants to browse from the information types of the candidates. it can. The type of information may be a hierarchical structure. For example, select “Mode for displaying information about food and drink”, and more specifically “Mode for displaying information about cake”, “Mode for displaying information about sushi”, “Mode for displaying information about pasta”, etc. It may be configured to embody the type of typical information.

  Here, it is assumed that the image display apparatus 101 is in a mode for searching for “cake” of “restaurant” according to the user's specification. FIG. 12 is a diagram illustrating an example of attribute information held by the wireless tags 103a to 103c. The wireless tag 103a includes, as attribute information associated with the identifier, information of “Suzuki cake shop: 50” (number “50” is the number viewed by another image display device) in the “restaurant” category. Have. The wireless tag 103b has information of “Sasaki cake: 15” in the “restaurant” category. The wireless tag 103c has information of “cake Yokoyama: 95” in the “restaurant” category.

  The determination unit 209 determines whether the attribute information includes information corresponding to the set type of information in order to determine whether the acquired attribute information is information desired by the user. In addition, the priority is determined for each attribute information according to the determination whether the information corresponding to the set type of information is included in the attribute information and the content of the attribute information. Here, the determination unit 209 determines that information related to the type of information having a large number of browsing is useful information to be presented to the user, and further increases the priority. Therefore, assuming that the attribute information corresponding to the identifiers of the wireless tags 103a to 103c is the attribute information shown in FIG. 12, all the information related to the user-desired “cake” exists. Subsequently, the determination unit 209 obtains the number of viewings of the attribute information determined to be information desired by the user, and the determination unit 209 determines that the attribute information has a large number of viewings, in this case, the wireless tag 103c and the wireless tag 103a. The priority is assigned in the order of the wireless tag 103b. In the case of attribute information that does not have user-desired information, the priority is set lower than attribute information that has user-desired information.

  Subsequently, the determination unit 208 determines a non-display region in which attribute information in the captured image is not displayed based on the object region detected by the detection unit 210 (step S505). Here, the determination unit 208 initially sets a region corresponding to the detected object in the captured image as a non-display region. The determination unit 208 calculates an area obtained by subtracting the sum of the object regions detected by the detection unit 210 from the entire region of the image, and divides the area by the number of detected subjects to display the attribute information display region for one subject The area of (area that can display attribute information that is not a non-display area) is calculated. Subsequently, the determination unit 208 determines the attribute information to be displayed according to the priority and the detected subject area (step S506). Here, the determination unit 208 determines the attribute information display number (n) that is the number of attribute information to be displayed by the following equation.

  Here, a is the area of the entire image area, i is the priority order of each attribute information, bi is the area of the subject area corresponding to each attribute information, n is the number of attribute information displays, and x is the number per attribute information. A constant value that is an area necessary for displaying attribute information is set. The determination unit 208 determines the number of pieces of attribute information to be displayed by obtaining the maximum n that satisfies the mathematical expression represented by the expression (1). For example, when the display area for one subject is equal to or greater than a certain value x that can display attribute information, it is determined that attribute information corresponding to all the subjects detected by the detection unit 210 is displayed. When the display area for one subject is less than or equal to a certain value x, the number of subjects for which the display area for one subject is greater than or equal to a certain value is calculated and set as the attribute information display number (n). In other words, when determining that all the attribute information cannot be displayed, the determination unit 208 determines that attribute information with a low priority is not displayed in order to preferentially display attribute information with a high priority. Further, the determination unit 208 determines that an area in the image of the subject corresponding to the attribute information that is not displayed is a display area in which the attribute information can be displayed. Through the above processing, the determination unit 208 determines the number of attribute information displays (step S506). For example, if the total area of the image is 100, the sum of the areas of the detected subject is 49, and the constant value x is 25, the maximum number of attribute information displays is 2. In this way, the number of attribute information displays can be determined.

  Subsequently, the determination unit 208 resets the area of the subject corresponding to the attribute information to be displayed as a non-display area (step S507). In other words, the determination unit 208 changes the setting so that the area of the subject corresponding to the attribute information not displayed from the non-display area set in S505 is set as the display area. The image display apparatus 101 defines a processing variable a indicating the order of priority, and sets its initial value as a = 1 (step S507). The image display apparatus 101 determines whether or not the display of all attribute information to be displayed has been completed by determining whether or not n ≧ a is true (step S509). If the image display apparatus 101 determines that n ≧ a is true, the process proceeds to step S510. If the image display apparatus 101 determines that n ≧ a is false, the image display apparatus 101 ends the process.

  The determination unit 208 determines an area for displaying the a-th priority attribute information (step S510). Here, the area for displaying the attribute information is preferably in the display area in the vicinity of the area in the subject image corresponding to the attribute information (within a predetermined distance from the subject area). The synthesizing unit 207 synthesizes the attribute information with the determined area and displays it (step S511).

  The determination unit 208 adds an area in the captured image that displays the attribute information displayed in step S511 to the non-display area. (Step S511). The image display apparatus 101 performs the process of a = a + 1 and lowers the priority order of the attribute information to be processed by one (step S513). The image display apparatus 101 repeats the processing of S509 to S513 until display of all the attribute information to be displayed is completed.

  Next, processing when changing the type of information set based on a user instruction in the image display apparatus 101 will be described with reference to the flowchart shown in FIG. The image display apparatus 101 detects a mode change (step S601). For example, the user or the system changes the mode in which recommended restaurants are preferentially displayed to the mode in which favorite music is preferentially displayed. As other modes, there are a person mode for preferentially browsing person attribute information and a building mode for preferentially browsing building attribute information. By executing these, it is possible to display attribute information without disturbing user-desired information by other subjects or other information not desired by the user.

  The image display apparatus 101 determines whether the mode has been changed (step S602). If the determination unit 209 determines that the mode is the same as the previously set mode and that the mode has not been changed, the process ends. If the determination unit 209 determines that the mode has been changed to a mode different from the previously set mode, the process proceeds to step S603. The image display device 101 re-executes image display. The image display process shown in FIG. 5 is executed. Before re-execution, the composition unit 207 deletes the displayed attribute information. In addition, since the process of S501-S503 is already performed, you may abbreviate | omit a process.

  Next, FIG. 7 is a diagram illustrating a sequence of the image display apparatus 101, the information management server 102, and the wireless tags 103a to 103c. The imaging unit 201 of the image display apparatus 101 captures an image (M701). The RF unit 212 of the image display apparatus 101 performs sensing of the wireless tag 103 (M702). The identifier acquisition unit 203 of the image display apparatus 101 makes an acquisition request for each tag ID (identifier) from the wireless tags 103a to 103c (M703 to M705). The identifier acquisition unit 203 of the image display apparatus 101 receives the respective tag IDs (identifiers) from the wireless tags 103a to 103c (M706 to M708) (S501 in FIG. 5).

  The feature information acquisition unit 204 of the image display apparatus 101 requests the information management server 102 for feature information corresponding to each identifier using the tag IDs (identifiers) of the wireless tags 103a to 103c as keys. The attribute information acquisition unit 205 of the image display apparatus 101 requests the attribute information corresponding to each identifier from the information management server 102 using the tag IDs (identifiers) of the wireless tags 103a to 103c as keys (M709) (FIG. 5). S502). In this embodiment, the request messages are combined into one. However, the present invention is not limited to this, and the feature information and attribute information may be transmitted separately, or may be transmitted for each tag ID (identifier). Further, in the present embodiment, there is one information management server that holds the feature information and the attribute information. However, the information management server is not limited to this. You may connect to an information management server. The transmission / reception unit 303 of the information management server 102 receives a request for feature information and a request for attribute information from the image display apparatus 101. The feature information holding unit 301 searches each associated feature information using each received tag ID (identifier) as a key, and transmits it to the image display apparatus 101. The feature information holding unit 301 searches the attribute information associated with each tag ID (identifier) using each tag ID (identifier) as a key, and transmits the attribute information to the image display apparatus 101 (M710). In this embodiment, the feature information and attribute information are transmitted together, but may be transmitted separately.

  The detection unit 210 of the image display apparatus 101 identifies an object from the captured image based on the feature information acquired from the information management server (M711) (S503 in FIG. 5). If the object cannot be specified, the related attribute information is not displayed and is not included in the attribute information display count. The determination unit 209 of the image display apparatus 101 determines the priority from the content of the attribute information of the identified object and the type of information set (M712, S504). The determination unit 208 of the image display apparatus 101 determines the number of attribute information displays (M713, S506). The determination unit 208 of the image display apparatus 101 sets only the number of attribute information displays determined in step S506 from the high priority subject area as a non-display area (M714, S507). The synthesizing unit 207 of the image display apparatus 101 synthesizes the attribute information with the captured image (S508 to S513). The display unit 202 of the image display apparatus 101 displays the synthesized image on the display (M715).

  Subsequently, an example for explaining an operation for displaying the attribute information on the display unit is shown in FIG. FIG. 8A is a captured image before the attribute information acquired by the imaging unit 201 is combined. The RF unit 212 and the identifier acquisition unit 203 recognize the wireless tags 103a to 103c included in the subject in the captured image (S501). Although the wireless tag is illustrated, the wireless tag does not need to be captured in the captured image. FIG. 8B shows a state where the detection unit 210 performs image recognition using feature information related to a tag ID (identifier) and detects a subject. Reference numeral 801 denotes a region in the image of the detected object. FIG. 8C shows a state where the area of the subject recognized by the determination unit 208 is set as a non-display area. Reference numeral 802 denotes a non-display area. FIG. 8D shows a state where the attribute information of each subject is displayed in the display area. Reference numeral 803 denotes attribute information combined with the captured image and displayed. In the example of FIG. 8, since all attribute information corresponding to the object in which the display area is detected can be displayed, the attribute information of all attribute information corresponding to the object is displayed. In addition, although the area | region was illustrated in order to demonstrate the area | region which detected the object in FIG.

  Next, FIG. 14 shows another example for explaining an operation for displaying the attribute information on the display unit. FIG. 14 is an example when the composition is changed to include a person who is an object associated with the wireless tags 103a to 103c shown in FIG. FIG. 14A shows a captured image before the attribute information acquired by the imaging unit 201 is combined. The RF unit 212 and the identifier acquisition unit 203 recognize the wireless tags 103a to 103c included in the subject in the captured image (S501). FIG. 14B shows a state where the detection unit 210 performs image recognition using feature information related to a tag ID (identifier) and detects a subject (S503). Reference numeral 1401 denotes a region in the image of the detected object. In FIG. 14B, the sum and difference of the entire area of the captured image and the area in the captured image corresponding to the detected object are not sufficient to display the attribute information of all the wireless tags 103a to 103c. It is determined. Then, it is determined that the attribute information display number n must be less than the detected number of objects. Here, it is assumed that the attribute information display number is determined to be 2 (S505). The determination unit 208 determines the attribute information to be displayed according to the priority according to the set type of information (S506). In this example, it is assumed that the priority is the lowest among the attribute information of the object corresponding to the identifier in which the attribute information associated with the wireless tag 103b is detected.

  FIG. 14C shows a state where the non-display area is reset according to the attribute information display count (n). Reference numeral 1402 denotes a non-display area, and an object area corresponding to attribute information having a high priority is set as a non-display old area (S507). Here, the area of the object corresponding to the wireless tag 103a and the wireless tag 103c is set as a non-display area. On the other hand, an object area 1403 corresponding to the wireless tag 103b having the lowest priority is set as a display area. FIG. 14D shows a state in which only attribute information determined to be displayed is displayed in the display area. 1404 is attribute information associated with the wireless tag 103a, and is displayed in association with the corresponding object. Moreover, it is attribute information displayed by being synthesized with the captured image. Reference numeral 1405 denotes attribute information associated with the wireless tag 103c, which is displayed in association with the corresponding object. The attribute information 1404 and the attribute information 1405 are displayed in the object area 1403 of the wireless tag 103b determined as non-displayed attribute information in the captured image. In addition, although the area | region was illustrated in FIG. 14 in order to demonstrate the area | region which detected the object, and the non-display area | region, it is not illustrated and displayed on an actual display part.

  Comparing FIG. 14D and FIG. 8D, the attribute information of the wireless tag 103b is displayed according to the area in the captured image of the detected object, although the same object exists in the captured image. You can see that it is gone. In other words, the attribute information to be displayed changes depending on whether or not there is an area other than the detected object area in the captured image that can display the attribute information of all the detected objects. If there is no area other than the object area corresponding to the detected identifier in the captured image as long as the attribute information of all the detected objects can be displayed, depending on the content of the attribute information corresponding to the detected object The attribute information with the higher priority set in the above is displayed with priority, and the attribute information with the lower priority is not displayed. In addition, attribute information having a high priority is displayed in an object area corresponding to attribute information having a low priority in the captured image.

  Next, FIG. 9 is a diagram illustrating a state where all the attribute information corresponding to the detected object cannot be displayed. FIG. 9A shows an example in which the detection unit 210 performs image recognition using feature information related to a tag ID (identifier) and detects 10 subjects. Reference numeral 901 denotes an area when the area in the image of the detected object is set as a non-display area. FIG. 9B is a diagram illustrating a state in which the priority of the attribute information corresponding to each subject is determined. Reference numeral 902 indicates the priority order of all detected objects. FIG. 9C is a diagram when the attribute information display number is determined to be three. Since the attribute information display number is 3, only the areas corresponding to the subjects having the priorities 1 to 3 are set as non-display areas, and the areas of the other subjects having the priorities 4 to 10 are set as the display areas. FIG. 9D shows a state in which subject attribute information is displayed in the display area. Reference numerals 903 to 905 denote attribute information that is combined with the captured image and displayed. In FIG. 9A, the attribute information 903 is displayed in the area set as the non-display area 901. As a result, the attribute information is displayed on the display unit 202 of the image display apparatus 101 at the position shown in FIG. In addition, although the area is illustrated in FIG. 9 for explaining the area where the subject is detected and the non-display area, it is not illustrated and displayed on the actual display unit.

  FIG. 13 is an example for explaining the operation when the user changes the type of information from the display screen of the image display apparatus 101 shown in FIG. FIG. 13A shows an example in which the detection unit 210 performs image recognition using feature information related to the tag ID (identifier) and detects 10 subjects, and is similar to FIG. 9A. is there. FIG. 13B is a diagram showing a state in which the priority of the attribute information corresponding to each changed subject is determined when the user changes the type of information. Comparing FIG. 9B and FIG. 13B, it can be seen that the priority order of the attribute information corresponding to each object is changed by the change in the setting of the information type by the user. Reference numeral 1302 indicates a changed priority order of all detected objects. FIG. 13C is a diagram when the attribute information display number is determined to be three. Since the attribute information display number is 3, only the areas corresponding to the subjects having the priorities 1 to 3 are set as the non-display areas, and the areas of the other subjects having the priorities 4 to 10 are set as the display areas. Comparing FIG. 13 (c) and FIG. 9 (c), FIG. 9 (d) shows that the attribute information of the subject is displayed in the display area by changing the setting of the information type. Reference numeral 903 denotes attribute information that is combined with the captured image and displayed. In FIG. 9A, the attribute information 903 is displayed in the area set as the non-display area 901. In addition, although the area | region was illustrated in order to demonstrate the area | region which detected the object in FIG. 13, and a non-display area | region, it is not illustrated and displayed on an actual display part.

  As described above, when there are too many objects to be displayed in association with attribute information in the image and all the attribute information cannot be displayed so as not to overlap the object, the user's desired information is given priority. Can be displayed. As a result, the user can browse the desired information in association with the object transmitting the information. In addition, by setting the area for displaying attribute information and the area for displaying the attribute information as non-display areas, it is possible to reduce the object from being hidden by displaying other attribute information. In addition, the information to be displayed can be prevented from overlapping. Further, since the priority is set according to the type of information set by the user, the user's desired information can be displayed with priority.

  In this embodiment, the area for displaying the attribute information is a non-display area. At this time, the display frame of the attribute information to be displayed (the part where the attribute information is not described) is changed to other attribute information as necessary. It is good also as a structure made into the semi-display area | region which is an area | region which can be displayed. For example, as indicated by 1001 in FIG. 10, the attribute information display areas may overlap each other. Before and after the overlap at this time is determined according to the priority. Moreover, you may determine according to other parameters | indexes, such as distance. Further, the vicinity of the detected subject may be set as the semi-display area.

  In step S502, feature information and attribute information are acquired simultaneously. Not limited to this, the image display apparatus 101 may acquire the attribute information of the identifier corresponding to the object detected by the detection unit 210 after acquiring the feature information first. Further, in this embodiment, the priority is determined based on the number of browsing included in the attribute information. However, the present invention is not limited to this, and the information management server 102 has the relationship between the image display device 101 and the wireless tag 103 (display rejection, It is also possible to determine and determine the relationship (such as display priority). Moreover, the priority is determined based on the relationship between the attribute information and the type of the selected information, but the priority is determined based on the relationship between the keyword and the attribute information input by the user instead of selecting the information type. I do not care. Also, in this embodiment, the system presents information type candidates to the user in advance, but the information type candidates to be presented are attributes associated with identifiers of wireless tags that exist within a range that can communicate with the image display device. Depending on the information, it may be configured to be variable. The configuration may be selected.

  In this embodiment, the priority is assigned to attribute information that does not have information related to the set information type. However, priority is given to attribute information that does not have information related to the information type. It is good also as a structure which does not allocate. For example, it is assumed that the type of information set by the image display apparatus 101 is a mode for searching for “fashion”. In FIG. 12, since only the wireless tag 103c has information related to “fashion”, priority is assigned only to attribute information corresponding to the wireless tag 103c. Therefore, only attribute information corresponding to the wireless tag 103c is displayed. Thus, since only the information desired by the user is displayed superimposed on the captured image, it is reduced that the visibility of the information due to unnecessary information is reduced, so that usability is improved.

(Other examples)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

DESCRIPTION OF SYMBOLS 101 Image display apparatus 102 Information management server 103 Wireless tag

Claims (13)

A display control device for displaying a captured image,
Identifier acquisition means for acquiring an identifier of another device;
Information acquisition means for acquiring information associated with the identifier;
Detection means for detecting an object corresponding to the identifier acquired by the identifier acquisition means from the captured image;
Selection means for selecting the information to be displayed according to the information acquired by the information acquisition means corresponding to the object detected by the detection means and the area of the object on the captured image detected by the detection means A display control device. A judgment unit for judging the information acquired by the information acquisition unit related to the type of information set by the user;
The display control apparatus according to claim 1, wherein the selection unit selects the information to be displayed based on a determination by the determination unit. A display control means for displaying the captured image and the information acquired by the information acquisition means selected by the selection means in a superimposed manner;
The display control apparatus according to claim 1, wherein the display control unit displays an object detected by the detection unit in association with information acquired by the information acquisition unit corresponding to the object.   The said display control means does not display the other information which the said information acquisition means acquired in the area | region on the picked-up image of the said object corresponding to the said information selected by the said selection means. Display controller.   The display control apparatus according to claim 3, wherein the display control unit displays the information to be displayed so as not to overlap each other.   6. The display control unit according to claim 3, wherein the display control unit displays the selected information when displayed in a region on the captured image of the object corresponding to the information not selected when displayed by the selection unit. The display control apparatus according to claim 1.   The display control apparatus according to claim 1, wherein the selection unit selects information to be displayed according to an area of an object on a captured image detected by the detection unit.   The selection unit is set by the user by the determination unit among the information acquired by the information acquisition unit corresponding to the object detected by the detection unit according to the area of the object on the captured image detected by the detection unit. 3. The display control apparatus according to claim 2, wherein information determined to be related to the type of information displayed is displayed, and information not determined to be related to the type of information set by the user by the determination unit is not displayed. . A determining unit that determines the number of pieces of information to be displayed on the captured image according to a region of the object on the captured image detected by the detecting unit;
The display control apparatus according to claim 1, wherein the selection unit selects the number of pieces of information determined by the determination unit. A display control device for displaying a captured image,
Identifier acquisition means for acquiring an identifier of another device;
Information acquisition means for acquiring information associated with the identifier from;
Detecting means for detecting an object corresponding to the identifier acquired by the identifier acquiring means from the captured image;
Information to be displayed among the information acquired by the information acquisition unit according to the information acquired by the information acquisition unit corresponding to the object detected by the detection unit and the area of the object on the captured image detected by the detection unit A display control device characterized in that changes. A control method of a display control device for displaying a captured image,
An identifier acquisition step of acquiring an identifier of another device;
An information acquisition step of acquiring information associated with the identifier;
A detection step of detecting an object corresponding to the identifier acquired by the identifier acquisition step from the captured image;
A selection step of selecting the information to be displayed according to the information acquired by the information acquisition step corresponding to the object detected by the detection step and the area of the object on the captured image detected by the detection step. A control method for a display control device. A control method of a display control device for displaying a captured image,
An identifier acquisition step of acquiring an identifier of another device;
An information acquisition step of acquiring information associated with the identifier;
A detection step of detecting an object corresponding to the identifier acquired by the identifier acquisition step from the captured image,
Information to be displayed among the information acquired by the information acquisition step according to the information acquired by the information acquisition step corresponding to the object detected by the detection step and the area of the object on the captured image detected by the detection step A control method for a display control device, wherein   The program for making a computer perform the control method of Claim 11 or 12.

Images