WO2022215347A1 - Système de commande d'affichage de contenu - Google Patents

Système de commande d'affichage de contenu Download PDF

Info

Publication number
WO2022215347A1
WO2022215347A1 PCT/JP2022/005304 JP2022005304W WO2022215347A1 WO 2022215347 A1 WO2022215347 A1 WO 2022215347A1 JP 2022005304 W JP2022005304 W JP 2022005304W WO 2022215347 A1 WO2022215347 A1 WO 2022215347A1
Authority
WO
WIPO (PCT)
Prior art keywords
content
user
display
target content
display control
Prior art date
Application number
PCT/JP2022/005304
Other languages
English (en)
Japanese (ja)
Inventor
有希 中村
康夫 森永
望 松本
達哉 西▲崎▼
怜央 水田
弘行 藤野
Original Assignee
株式会社Ｎｔｔドコモ
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社Ｎｔｔドコモ filed Critical 株式会社Ｎｔｔドコモ
Publication of WO2022215347A1 publication Critical patent/WO2022215347A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/38—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory with means for controlling the display position

Definitions

  • the present disclosure relates to a content display control system that controls content display on a head-mounted display such as AR (Augmented Reality) glasses.
  • a head-mounted display such as AR (Augmented Reality) glasses.
  • various contents for example, browser screens, display screens for various information provided by cloud services, etc., image display screens, video display screens, etc. etc.
  • various contents for example, browser screens, display screens for various information provided by cloud services, etc., image display screens, video display screens, etc. etc.
  • the present disclosure has been made to solve the above problems, and aims to arrange desired content at a position in the virtual space that is easy for the user to see without taking time and effort.
  • a content display control system includes a head-mounted display worn by a user to display content in a virtual space, a detection unit that detects a display instruction operation associated with target content reproduced by the user, and a control unit that controls content display on the head-mounted display such that the target content associated with the display instruction operation detected by the detection unit is displayed at the line-of-sight position of the user in the virtual space.
  • the ⁇ position of the user's line of sight'' here means (1) the focal position of the user's line of sight, (2) the extension line passing through the reticle in the 360-degree omnidirectional virtual space centered on the user, and the omnispherical surface. and (3) a position pointed by a user using an operating device (controller).
  • the control unit controls the target content associated with the detected display instruction action. controls the display of content on the head-mounted display so that is displayed at the line-of-sight position of the user in the virtual space.
  • desired content can be arranged at a position in the virtual space that is easy for the user to see without taking time and effort.
  • a content display control system includes a head-mounted display worn by a user to display content in a virtual space, and a detection unit that detects a display instruction operation that is reproduced by the user and is associated with target content.
  • a control unit that controls content display on the head-mounted display such that the target content associated with the display instruction operation detected by the detection unit is displayed at a position suitable for visual recognition by the user in a virtual space; , provided.
  • the "user's preferred viewing position" includes a position in front of the face that does not depend on the user's line of sight.
  • FIG. 4 is a flow diagram showing a rough flow of processing executed in the content display control system;
  • (a) is a diagram showing an example of tagging and registration of content in a virtual space, and
  • (b) is a diagram showing an example of tagging and registration of content from a dedicated management system.
  • FIG. 10 is a flow diagram showing processing related to display control of target content at the line-of-sight position of the user; (a) is a diagram showing the initial state when the tag of the target content is uttered, (b) is a diagram showing a comparison between the speech recognition result and the correspondence table, and (c) is the state after rearrangement.
  • FIG. 4 is a diagram showing; (a) is a diagram showing rearrangement pattern 1 for simple superimposition of target content, (b) is a diagram showing rearrangement pattern 2 for temporary placement of target content, and (c) is a diagram showing rearrangement pattern 2 for temporary placement of target content.
  • FIG. 10D is a diagram showing a rearrangement pattern 3 for simple superimposition and holding of relative positional relationships of all contents in the azimuth direction, and FIG. It is a figure for demonstrating conversion between a rectangular coordinate and a three-dimensional polar coordinate (spherical surface coordinate).
  • FIG. 4 is a diagram for explaining a first algorithm for rearranging target content to a line-of-sight position;
  • FIG. 10 is a diagram showing a calculation example according to the first algorithm in rearrangement pattern 1;
  • FIG. 10 is a diagram for explaining a second algorithm for rearranging target content to a line-of-sight position;
  • FIG. 10 is a diagram for explaining an algorithm for maintaining a relative positional relationship in the azimuth direction in rearrangement pattern 3;
  • FIG. 11 is a diagram for explaining an algorithm for maintaining relative positional relationships in rearrangement pattern 4;
  • FIG. 11 is a diagram for explaining an algorithm for maintaining relative positional relationships in rearrangement pattern 4;
  • FIG. 11 is a diagram for explaining an algorithm for maintaining relative positional relationships in rearrangement pattern 4;
  • FIG. 11 is a diagram for explaining an algorithm for maintaining relative positional relationships in rearrangement pattern 4;
  • (a) is a diagram showing semi-transparency of non-targeted content
  • (b) is a diagram showing sharpness adjustment of non-targeted content
  • (c) is a diagram showing outline glow adjustment of targeted content
  • (d) is a diagram showing target content display on a pop-up virtual screen.
  • FIG. 10 is a flow diagram showing processing related to display control of target content in front of a user's face.
  • (a) is a diagram showing a state when the tag of the target content is uttered while directing the line of sight in a direction different from the direction of the face, and
  • (b) is a diagram showing the state in which the target content is displayed at a display position in front of the face.
  • FIG. 4C is a diagram showing a state
  • (c) is a diagram showing a state when the line of sight is turned to the target content in front of the face.
  • the content display control system 1 includes a controller 20 operated by a user, a terminal 10 carried by the user, and AR glasses 30 worn by the user. Prepare.
  • the controller 20 has, for example, a configuration including buttons 21, a touch pad 22, a trigger 23 and a bumper 24 shown in FIG. 2(a).
  • the controller 20 is not an essential requirement in the content display control system 1, but can be replaced by the user interface of the terminal 10.
  • FIG. 2(b) the user interface of the terminal 10 shown in FIG. 2(b) can be substituted by a user interface including a touch pad 16 consisting of a total of five buttons in the center, up, down, left, and right, and buttons 17 and 18.
  • the AR glasses 30 correspond to a head-mounted display that displays content in virtual space, and displays content in a 360-degree omnidirectional virtual space centered on the user wearing it.
  • AR glasses are exemplified as a head-mounted display, but head-mounted displays other than AR glasses (for example, VR (Virtual Reality) goggles, etc.) may be employed.
  • VR Virtual Reality
  • the terminal 10 corresponds to, for example, a smart phone, a mobile phone, a notebook personal computer, etc., and includes a detection unit 11 and a control unit 12 shown in FIG. 1(a). The function, operation, etc. of each unit will be described below.
  • the detection unit 11 is a functional unit that detects a display instruction operation that is reproduced by the user and is associated with the target content.
  • a display instruction operation of uttering a character string (tag) indicating target content while performing a predetermined operation using the controller 20 will be described.
  • a predetermined operation using the controller 20 here will be exemplified later.
  • the control unit 12 is a functional unit that controls content display on the AR glasses 30 so that the target content associated with the display instruction operation detected by the detection unit 11 is displayed at the user's line-of-sight position in the virtual space.
  • the control unit 12 includes a specifying unit 12A and a display control unit 12B as described below.
  • the specifying unit 12A is a functional unit that specifies the target content associated with the display instruction operation detected by the detecting unit 11.
  • the display control unit 12B acquires the line-of-sight position of the user and controls content display on the AR glasses 30 so that the target content identified by the identification unit 12A is displayed at the line-of-sight position of the user in the virtual space. Department.
  • the ⁇ position of the user's line of sight'' includes (1) the focal position of the user's line of sight, (2) the extension line passing through the reticle in the 360-degree omnidirectional virtual space centered on the user, and the omnidirectional surface. and (3) the position pointed by the user using the controller 20. Of these, when (3) is adopted, the content using the controller 20 is used for the display instruction operation by the user. A pointing operation for the display position is added.
  • a character string (tag) indicating the target content is linked in advance to the target content as "content specifying information" for specifying the target content.
  • the user designates the target content by uttering a tag, which is a character string indicating the target content, while performing a predetermined operation described later using the controller 20 . Therefore, the detection unit 11 includes a microphone 11A that collects the user's utterance, and a voice recognition unit 11B that recognizes and converts the collected voice into text.
  • the detection unit 11 performs a display instruction operation such as "uttering a character string (tag) indicating the target content while performing a predetermined operation using the controller 20" (that is, a display instruction action) is detected, the specifying unit 12A specifies the target content linked to the content specifying information (character string (tag)) corresponding to the detected display instruction action.
  • a display instruction operation such as "uttering a character string (tag) indicating the target content while performing a predetermined operation using the controller 20" (that is, a display instruction action) is detected
  • the specifying unit 12A specifies the target content linked to the content specifying information (character string (tag)) corresponding to the detected display instruction action.
  • the target content is specified by a gesture of a part of the user's body (for example, the user's hand). and a gesture recognition unit 11D for recognizing gestures from moving image data obtained by imaging.
  • the predetermined actions using the controller 20 include, for example, the action of continuously pressing the button 21 or the bumper 24 n times, the action of long-pressing the button 21 or the bumper 24, the action of holding the trigger 23 while pulling it for a certain period of time, Examples include an action of tapping a specific portion of the touch pad 22 at the top, bottom, left, right, and center n times consecutively, and an action of pressing and holding a specific portion of the touch pad 22 at the top, bottom, left, right, and center.
  • the "long press action” various variations can be provided by combining a plurality of actions as described above, such as starting a long press after tapping n times.
  • the terminal 10 stores a correspondence table 13 in which content identification information for various contents is linked to content IDs for identifying the contents, and a user ID.
  • a registration unit 14 may further include a registration unit 14 for linking the content identification information to the content ID and registering it in the correspondence table 13 by performing a predetermined registration process.
  • the correspondence table 13 is a content correspondence information database in which content identification information (for example, a tag consisting of a character string, a controller operation pattern, a hand gesture, etc.) is stored in association with a content ID, and an example is shown in FIG. As shown, tags associated with various content IDs are registered. In the following, processing operations, effects, etc. will be described assuming that the terminal 10 has the configuration shown in FIG. 1(b).
  • the processing executed in the content display control system 1 includes tagging registration to content (step S1) and display control of target content at the user's line of sight position (step S2).
  • the tagging registration in step S1 is a process that is performed before the process in step S2, which is a feature of the present disclosure.
  • the user operates the controller 20 to expand the menu of the content, select "tagging registration", and enter the tag (character string) of the content to tag the content. and register.
  • the tag "ALPACA" is registered in the correspondence table 13.
  • the content may be tagged and registered from a dedicated management system running on a personal computer.
  • step S2 in FIG. 3 the display control of the target content at the line-of-sight position of the user (step S2 in FIG. 3) will be described.
  • Execution of the processing in FIG. 6 is started when the user utters a tag, which is a character string indicating the target content, while performing a predetermined action using the controller 20 .
  • the detection unit 11 collects the user's utterance with the microphone 11A, and the speech recognition unit 11B performs speech recognition on the collected speech (step S21).
  • the text data of the speech recognition result is transferred to the identification unit 12A.
  • the identification unit 12A compares the speech recognition result with each record stored in the correspondence table 13 (step S22), and determines whether or not there is a record that matches the speech recognition result (step S23). If there is no record that matches the speech recognition result, the process is terminated. On the other hand, if there is a record that matches the voice recognition result, the specifying unit 12A specifies the content of the matching record as the target content, and transmits it to the display control unit 12B. Then, the display control unit 12B determines whether or not the target content is being displayed in the virtual space (step S24). Here, if the target content is being displayed, the display control unit 12B acquires the line-of-sight position of the user (step S25), and rearranges the content based on the line-of-sight position (step S25). S26).
  • the display control unit 12B acquires the line-of-sight position of the user (step S27), renders the target content at the line-of-sight position, and displays the target content at the line-of-sight position. (Step S28).
  • the initial state is a state in which four types of animal content are arranged with the user at the center in the virtual space.
  • the detection unit 11 recognizes the voice of the tag "FOX” by voice recognition, obtains the text "FOX” as a voice recognition result, and transfers it to the identification unit 12A.
  • the identifying unit 12A compares the text "FOX" with the tag of each record stored in the correspondence table 13, and as shown in FIG.
  • step S23 the content of the matching record (the content of the fox with the content ID "ID-JSN3G49”) is specified as the target content, and the display is controlled. It is transmitted to the section 12B. Further, the display control unit 12B determines that the target content is being displayed in the virtual space (step S24), and sets the user's gaze position ((x e , y e , ze ) shown in FIG. 7A) to Acquired (step S25), and rearrangement of the contents based on the line-of-sight position is executed according to a method described later (step S26). As an execution example of rearrangement, FIG. 7(c) shows an example in which only the fox content, which is the target content, is simply rearranged to the user's line-of-sight position (x e , y e , z e ).
  • the initial state before the rearrangement is a state in which four types of animal content are arranged around the user in the disguised space shown in FIG. 7A.
  • FIG. 8(a) shows a rearrangement pattern 1 that simply superimposes the target content.
  • This rearrangement pattern 1 simply rearranges only the target content (fox content in this case) to the user's gaze position (x e , y e , z e ) as in the example of FIG. 7(c). It's a pattern.
  • FIG. 8(b) shows rearrangement pattern 2 for temporarily arranging the target content.
  • this rearrangement pattern 2 only the target content (the fox content here) is temporarily moved to the user's gaze position (x e , y e , z e ), a copy of the target content is generated, and the copy is moved. It is a pattern that is temporarily placed at the original position (x t , y t , z t ). A copy of the target content is displayed, for example, translucent and cannot be operated.
  • the target content displayed at the user's line-of-sight position (x e , y e , z e ) is deleted, and the content displayed at the original position (x t , y t , z t ) is restored. is returned to the original display form, and the operation details, changes, etc. to the target content displayed at the user's line-of-sight position (x e , y e , z e ) are reflected.
  • FIG. 8(c) shows a rearrangement pattern 3 that simply superimposes the target content and maintains the relative positional relationship of all the content in the azimuth direction.
  • this rearrangement pattern 3 all of the displayed contents (hereinafter referred to as "display contents”) (here, four types of animals) are moved while maintaining the relative positional relationship between the contents in the azimuth direction, Furthermore, it is a pattern in which only the target content (here, fox content) is translated to the user's gaze position (x e , y e , z e ).
  • FIG. 8(d) shows a rearrangement pattern 4 that maintains the relative positional relationship between all displayed contents.
  • this rearrangement pattern 4 all the displayed contents are moved while maintaining the relative positional relationship between them, and the target contents (here, fox contents) are moved to the user's gaze position (x e , y e , z e ).
  • the target contents here, fox contents
  • the first algorithm is a procedure that rotates the target content around the Z-axis in Cartesian coordinates and translates the target content to the user's line-of-sight position.
  • FIG. 10 first, three-dimensional polar coordinates are obtained from rectangular coordinates for each of the target content and the user's line-of-sight position (process (1)).
  • FIG. 9 shows a general relationship between orthogonal coordinates and three-dimensional polar coordinates (spherical coordinates). , ⁇ ), the three-dimensional polar coordinates (r t , ⁇ t , ⁇ t ) of the target content and the three-dimensional polar coordinates (r e , ⁇ e , ⁇ e ). It should be noted that only the azimuth angle of the three-dimensional polar coordinates may be obtained.
  • Equation (3) the target content is rotated by ⁇ around the Z-axis in the orthogonal coordinates according to the following equation (3) (process (3)). That is, the azimuth angle with respect to the user is adjusted.
  • the first term on the right side of Equation (3) is a rotation matrix about the Z-axis.
  • Equation (4) the first term on the right side of Equation (4) is a translation matrix.
  • the orthogonal coordinates (x e , y e , z e ) of the user's gaze position are , the following is obtained as the three-dimensional polar coordinates (r e , ⁇ e , ⁇ e ) of the line-of-sight position of the user by substituting into the above formula (1).
  • the target content is rotated by ⁇ around the Z-axis in the orthogonal coordinates as follows (process (3)).
  • the target content is translated to the line-of-sight position of the user as follows (process (4)).
  • the Cartesian coordinates (x' t , y' t , z' t ) of the target content are the Cartesian coordinates (x e , y e , ze ) of the line-of-sight position of the user, that is, becomes.
  • the second algorithm is a procedure that rotates the target content about the Z-axis in Cartesian coordinates, adjusts the polar angle with respect to the user, and translates the target content to the user's gaze position.
  • first, three-dimensional polar coordinates are obtained from orthogonal coordinates for each of the target content and the user's line-of-sight position (process (1)). This process is similar to the first algorithm described above.
  • Equation (5) the target content is rotated by ⁇ around the Z-axis in the orthogonal coordinates according to the following equation (5) (process (3)). That is, the azimuth angle with respect to the user is adjusted.
  • the first term on the right side of Equation (5) is a rotation matrix about the Z-axis.
  • Equation (6) The first term on the right side of Equation (6) is the Rodriguez rotation matrix.
  • Equation (7) the orthogonal coordinates (x' t , y' t , z' t ) of the target content become the orthogonal coordinates (x e , y e , z e ) of the user's gaze position.
  • the first term on the right side of Equation (7) corresponds to the following.
  • each of all the display contents is rotated by ⁇ around the Z-axis in the orthogonal coordinates according to the following formula (8) (process (3)). That is, the azimuth angle with respect to the user is adjusted for each display content.
  • n is a subscript for specifying each display content.
  • the target content is translated to the user's line-of-sight position using Equation (4) described above, thereby relocating the target content to the user's line-of-sight position (processing ( Four)).
  • the rearrangement method may employ the method of the second algorithm instead of the method of the first algorithm.
  • Equation (10) is used to provisionally calculate the post-rearrangement azimuth angle of the display content other than the target content (process (4)).
  • m is a subscript for specifying content other than the target content.
  • the post-rearrangement polar angles of the contents other than the target contents among the displayed contents are tentatively calculated as follows (processing (5) shown in FIGS. 14 and 15). However, it is limited to the following range.
  • Azimuth angle ⁇ 'm (pre) after rearrangement is - ⁇ 'm (pre) ⁇ t +( ⁇ /2) or ⁇ -Arctan( -xtz / ytz ) ⁇ 'm If (pre) ⁇ ⁇ , the rearranged polar angle ⁇ 'm (pre) is obtained by the following equation (11).
  • ⁇ ' m (pre) ⁇ m + ⁇ (11)
  • ⁇ ' m (pre) ⁇ m + ⁇ (13)
  • the rearranged polar angle ⁇ ′ m(pre) is obtained by the following equation (14).
  • ⁇ ' m (pre) ⁇ m - ⁇ (14)
  • Azimuth angle ⁇ ' m (pre) after rearrangement is - ⁇ ⁇ ' m (pre) ⁇ - ⁇ + Arctan(-x tz /y tz ) or Arctan (-x tz /y tz ) ⁇ ⁇ ' m If (pre) ⁇ ⁇ , the rearranged polar angle ⁇ 'm (pre) is obtained by the following equation (15).
  • ⁇ ' m (pre) ⁇ m + ⁇ (15)
  • ⁇ ' m (pre) ⁇ m - ⁇ (16)
  • the user After determining the polar coordinates after rearrangement for content other than the target content as described above, the user presets TH (threshold value) as follows, and sets the radius r that defines the display area of the content in the depth direction. By adjusting, the adjusted radius r' m may be obtained. In this case, it is possible to prevent the radius r from becoming extremely small and the radius r from becoming extremely large due to the amount of movement of the target content. However, care must be taken because it can be regarded as a process that creates a shift in the depth direction and destroys the relative position.
  • TH threshold value
  • the coordinates after adjustment of the azimuth angle and polar angle obtained by the above processing are translated by applying the following equation (28), and the orthogonal coordinates after translation (x' m , y' m ,z' m ) is obtained (process (7) iv).
  • the Cartesian coordinates (x' m , y' m , z' m ) after rearrangement of all contents other than the target contents are obtained.
  • the target content in the virtual space is displayed according to a predetermined rearrangement pattern (any of the rearrangement patterns 1 to 4) based on the line-of-sight position.
  • a rearrangement position of the content can be determined, and the target content can be rearranged to the determined rearrangement position.
  • Target content can be displayed.
  • a rearrangement pattern for rearranging all the display contents while maintaining the relative positional relationship in the azimuth direction for all the display contents 3 it is possible to maintain the relative positional relationship in the azimuth angle direction for all display contents, and to avoid the layout that the user is particular about from collapsing significantly.
  • the rearrangement pattern 4 for rearranging all the display contents while maintaining the relative positional relationship between all the display contents is included. , the relative positional relationship between all displayed contents can be maintained, and it is possible to avoid collapsing of the user's preferred layout.
  • Modification 18(a) to 18(d) will be used to explain how to deal with overlap in content display. If there is a possibility that the target content and other content (non-target content) overlap in the virtual space, for example, by taking the following measures, the overlap can be avoided and the inconvenience of the target content becoming difficult to see can be resolved. can.
  • the non-target content is translucent, as shown in FIG. 18(a).
  • a method of making the target content relatively easy to see by performing processing such as adjusting the value of r may be adopted.
  • processing such as adjusting the brilliance of the outline of the target content may be performed to make the target content stand out and be easier to see.
  • the target content may be displayed on a pop-up virtual screen as shown in FIG. 18(d). In this way, various techniques can be used to make target content relatively easy to see.
  • FIG. 18D is limited to rearrangement pattern 2, and the processing for adjusting the value of radius vector r in spherical coordinates is limited to rearrangement patterns 1-3.
  • the user may specify the target content by inputting a pattern described later using a controller.
  • a hand gesture may be used to specify the target content.
  • the former "pattern using a controller”
  • FIG. a pattern in which the button 21 is pressed n times consecutively, and a pattern in which a laser pointer function of the controller 20 draws a specific figure (for example, a circle) in the virtual space.
  • the controller pattern as described above is registered in the correspondence table 13 in association with the content ID of the corresponding content.
  • Hand gestures for designating target content by the latter hand gesture include, as shown in FIG. be done.
  • the hand gesture as described above is registered in the correspondence table 13 in association with the content ID of the corresponding content. It should be noted that when target content is designated by a hand gesture, as described above, the detection unit 11 shown in FIGS.
  • the configuration further includes a gesture recognition unit 11D for recognition.
  • the user instead of uttering the tag of the target content, the user can specify the target content and instruct to display it by a method such as pattern input using a controller or hand gesture. Display instructions of desired content can be easily given by various methods.
  • the terminal 10 has a correspondence table 13 in which content specifying information (for example, a tag) is associated with a content ID and stored in advance, and the specifying unit 12A
  • content specifying information for example, a tag
  • the specifying unit 12A an example of specifying the target content from the content ID linked to the content specifying information corresponding to the display instruction operation detected by the detection unit 11 by referring to the correspondence table 13 has been shown.
  • the correspondence table 13 is not an essential requirement. For example, by using the content ID itself as a tag that is content identification information, the correspondence table 13 is made unnecessary, and the identification unit 12A identifies content corresponding to the detected display instruction operation.
  • the configuration shown in FIG. 1(a) may be adopted so that the target content can be specified immediately from the information. According to the configuration of FIG. 1(a), the simplification of the device configuration can be achieved.
  • the terminal 10 includes all of the illustrated components (the detection unit 11, the identification unit 12A, the display control unit 12B, etc.).
  • a part of the functional units may be installed in the server, and the target content may be specified by requesting the server from the terminal 10 to process.
  • the content display control system according to the present disclosure is understood to have a configuration including the terminal 10 and the server.
  • the display control unit 12B acquires the orientation of the user's face using an existing face recognition technology or the like, and the target content is displayed at a display position in front of the orientation of the user's face in the virtual space. , which controls the display of content on the head-mounted display.
  • display position in front of the face for example, the position of the intersection of the extension line to the front of the face in the 360-degree omnidirectional virtual space centered on the user and the omnidirectional surface is adopted.
  • a position at a predetermined distance from the face may be adopted on the forward extension of the face. It should be noted that the "predetermined distance" (distance from the face) can be appropriately adjusted by the user.
  • the processing shown in FIG. 21 is executed instead of the "display control to the user's gaze position" in FIG. 6 described above.
  • the same processing as in FIG. 6 is assigned the same number, and redundant description is omitted.
  • the display control unit 12B acquires the orientation of the user's face using existing face recognition technology or the like (step S25A). Content rearrangement, which will be described later, is executed based on the display position of (step S26A).
  • the display control unit 12B acquires the orientation of the user's face (step S27A), and renders the target content at a display position in front of the face orientation to display the target content.
  • the target content is displayed at the position (step S28A).
  • the initial state is a state in which three types of animal content are arranged with the user at the center in the virtual space.
  • the detection unit 11 recognizes the voice of the tag "ALPACA” in step S21 of FIG.
  • the text "ALPACA” is obtained as a speech recognition result and transferred to the identification unit 12A.
  • step S22 the specifying unit 12A compares the text "ALPACA" with the tag of each record stored in the correspondence table 13, and determines that there is a record matching the text "ALPACA" in the correspondence table 13 (step S22). YES in S23), the content of the matching record, Alpaca, is specified as the target content and transmitted to the display control unit 12B. Further, the display control unit 12B determines that the target content is being displayed in the virtual space (step S24), acquires the orientation of the user's face using existing face recognition technology or the like (step S25A), The content rearrangement is executed with reference to the forward display position ((x e , y e , z e ) shown in FIG.
  • FIG. 22B shows an example of rearranging only the target content (alpaca content) to the display position (x e , y e , z e ) in front of the face direction.
  • the user's line of sight is directed to the position before the movement of the target content, but the target content first moves to the display position in front of the face.
  • the user can view the target content at a position that is easy for the user to see (here, in front of the face) by directing the line of sight forward.
  • the display control unit 12B acquires the orientation of the user's face using an existing face authentication technology or the like (step S27A), and determines the display position in front of the orientation of the face.
  • the target content is displayed in front of the face (step S28A).
  • the target content can be displayed in front of the user's face regardless of the line of sight of the user.
  • each functional block may be implemented using one device that is physically or logically coupled, or directly or indirectly using two or more devices that are physically or logically separated (e.g. , wired, wireless, etc.) and may be implemented using these multiple devices.
  • a functional block may be implemented by combining software in the one device or the plurality of devices.
  • Functions include judging, determining, determining, calculating, calculating, processing, deriving, investigating, searching, checking, receiving, transmitting, outputting, accessing, resolving, selecting, choosing, establishing, comparing, assuming, expecting, assuming, Broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating, mapping, assigning, etc. can't
  • a functional block (component) that makes transmission work is called a transmitting unit or transmitter.
  • the implementation method is not particularly limited.
  • FIG. 20 is a diagram illustrating a hardware configuration example of terminal 10 according to an embodiment of the present disclosure.
  • the terminal 10 described above may be physically configured as a computer device including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, and the like.
  • the term "apparatus” can be read as a circuit, device, unit, or the like.
  • the hardware configuration of the terminal 10 may be configured to include one or more of each device shown in the figure, or may be configured without some of the devices.
  • Each function of the terminal 10 is performed by causing the processor 1001 to perform calculations, controlling communication by the communication device 1004, controlling communication by the communication device 1004, and controlling the communication by the memory 1002 and the It is realized by controlling at least one of data reading and writing in the storage 1003 .
  • the processor 1001 for example, operates an operating system and controls the entire computer.
  • the processor 1001 may be configured by a central processing unit (CPU) including interfaces with peripheral devices, terminals, arithmetic units, registers, and the like.
  • CPU central processing unit
  • the processor 1001 reads programs (program codes), software modules, data, etc. from at least one of the storage 1003 and the communication device 1004 to the memory 1002, and executes various processes according to them.
  • programs program codes
  • software modules software modules
  • data etc.
  • the program a program that causes a computer to execute at least part of the operations described in the above embodiments is used.
  • FIG. Processor 1001 may be implemented by one or more chips. Note that the program may be transmitted from a network via an electric communication line.
  • the memory 1002 is a computer-readable recording medium, and is composed of at least one of, for example, ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), etc. may be
  • ROM Read Only Memory
  • EPROM Erasable Programmable ROM
  • EEPROM Electrical Erasable Programmable ROM
  • RAM Random Access Memory
  • the memory 1002 may also be called a register, cache, main memory (main storage device), or the like.
  • the memory 1002 can store executable programs (program code), software modules, etc. for implementing a wireless communication method according to an embodiment of the present disclosure.
  • the storage 1003 is a computer-readable recording medium, for example, an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disc, a magneto-optical disc (for example, a compact disc, a digital versatile disc, a Blu-ray disk), smart card, flash memory (eg, card, stick, key drive), floppy disk, magnetic strip, and/or the like.
  • Storage 1003 may also be called an auxiliary storage device.
  • the storage medium described above may be, for example, a database, server, or other suitable medium including at least one of memory 1002 and storage 1003 .
  • the communication device 1004 is hardware (transmitting/receiving device) for communicating between computers via at least one of a wired network and a wireless network, and is also called a network device, a network controller, a network card, a communication module, or the like.
  • the input device 1005 is an input device (for example, keyboard, mouse, microphone, switch, button, sensor, etc.) that receives input from the outside.
  • the output device 1006 is an output device (eg, display, speaker, LED lamp, etc.) that outputs to the outside. Note that the input device 1005 and the output device 1006 may be integrated (for example, a touch panel).
  • Each device such as the processor 1001 and the memory 1002 is connected by a bus 1007 for communicating information.
  • the bus 1007 may be configured using a single bus, or may be configured using different buses between devices.
  • notification of predetermined information is not limited to being performed explicitly, but may be performed implicitly (for example, not notifying the predetermined information). good too.
  • Input/output information may be stored in a specific location (for example, memory) or managed using a management table. Input/output information and the like can be overwritten, updated, or appended. The output information and the like may be deleted. The entered information and the like may be transmitted to another device.
  • a and B are different may mean “A and B are different from each other.”
  • the term may also mean that "A and B are different from C”.
  • Terms such as “separate,” “coupled,” etc. may also be interpreted in the same manner as “different.”

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • User Interface Of Digital Computer (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

La présente invention concerne ce système de commande d'affichage de contenu 1 qui comprend : Des lunettes de réalité augmentée (AR) 30, sous la forme d'un visiocasque qui est porté par un utilisateur et affiche un contenu dans un espace virtuel; une unité de détection 11 servant à détecter une opération d'instruction d'affichage qui est reproduite par l'utilisateur et qui est associée à un contenu cible; et une unité de commande 12 servant à commander un affichage de contenu sur les lunettes d'AR 30 de sorte que le contenu cible associé à l'opération d'instruction d'affichage détectée est affiché à une position appropriée pour la reconnaissance visuelle de l'utilisateur dans un espace virtuel.
PCT/JP2022/005304 2021-04-08 2022-02-10 Système de commande d'affichage de contenu WO2022215347A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021-065976 2021-04-08
JP2021065976A JP2024098511A (ja) 2021-04-08 2021-04-08 コンテンツ表示制御システム

Publications (1)

Publication Number Publication Date
WO2022215347A1 true WO2022215347A1 (fr) 2022-10-13

Family

ID=83546319

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/005304 WO2022215347A1 (fr) 2021-04-08 2022-02-10 Système de commande d'affichage de contenu

Country Status (2)

Country Link
JP (1) JP2024098511A (fr)
WO (1) WO2022215347A1 (fr)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019095936A (ja) * 2017-11-20 2019-06-20 キヤノン株式会社 画像処理装置、画像処理方法およびプログラム
JP2020519986A (ja) * 2017-04-19 2020-07-02 マジック リープ， インコーポレイテッドＭａｇｉｃ Ｌｅａｐ，Ｉｎｃ． ウェアラブルシステムのためのマルチモード実行およびテキスト編集

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2020519986A (ja) * 2017-04-19 2020-07-02 マジック リープ， インコーポレイテッドＭａｇｉｃ Ｌｅａｐ，Ｉｎｃ． ウェアラブルシステムのためのマルチモード実行およびテキスト編集
JP2019095936A (ja) * 2017-11-20 2019-06-20 キヤノン株式会社 画像処理装置、画像処理方法およびプログラム

Also Published As

Publication number Publication date
JP2024098511A (ja) 2024-07-24

Similar Documents

Publication Publication Date Title
US10514758B2 (en) Visibility improvement method based on eye tracking, machine-readable storage medium and electronic device
US10732725B2 (en) Method and apparatus of interactive display based on gesture recognition
US20210026457A1 (en) Sharing Across Environments
US8135440B2 (en) System for using mobile communication terminal as pointer and method and medium thereof
JP3834766B2 (ja) マンマシーン・インターフェース・システム
KR102059913B1 (ko) 태그 정보 저장 방법 및 장치와 태그 정보를 이용한 이미지 검색 방법 및 장치
JP5844288B2 (ja) 機能拡張装置、機能拡張方法、機能拡張プログラム、及び集積回路
KR102285915B1 (ko) 모바일 디바이스를 위한 실시간 3d 제스처 인식 및 트랙킹 시스템
US9064436B1 (en) Text input on touch sensitive interface
KR102665643B1 (ko) 아바타 표시를 제어하기 위한 방법 및 그 전자 장치
JP2011217098A (ja) 情報処理システム、会議管理装置、情報処理方法、会議管理装置の制御方法及びプログラム
JP2003256142A (ja) 情報処理装置、情報処理プログラム、情報処理プログラムを記録したコンピュータ読み取り可能な記録媒体、及び情報処理方法
US20220374098A1 (en) Customizing user interfaces of binary applications
CN109189879A (zh) 电子书籍显示方法及装置
TWI703492B (zh) 控制使用者介面之方法、程式及裝置
US11620414B2 (en) Display apparatus, display method, and image processing system
CN110738185B (zh) 表单对象的识别方法、装置及存储介质
JP5342806B2 (ja) 表示方法および表示装置
WO2022215347A1 (fr) Système de commande d'affichage de contenu
KR102570009B1 (ko) Ar 객체 생성 방법 및 전자 장치
KR20190102479A (ko) 이동단말기 및 그 제어 방법
KR102294717B1 (ko) 변형 오브젝트를 제공하기 위한 증강현실 영상 제공 시스템 및 방법
JP2022525326A (ja) ２次元カメラを利用して客体制御を支援する方法、システムおよび非一過性のコンピュータ読み取り可能な記録媒体
JP6208910B1 (ja) 動画像処理装置、動画像処理システム、動画像処理方法及び動画像処理プログラム
JP2009015720A (ja) 認識装置および認識方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22784332

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22784332

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP