WO2024029275A1 - 表示制御システム - Google Patents

表示制御システム Download PDF

Info

Publication number
WO2024029275A1
WO2024029275A1 PCT/JP2023/025317 JP2023025317W WO2024029275A1 WO 2024029275 A1 WO2024029275 A1 WO 2024029275A1 JP 2023025317 W JP2023025317 W JP 2023025317W WO 2024029275 A1 WO2024029275 A1 WO 2024029275A1
Authority
WO
WIPO (PCT)
Prior art keywords
file
virtual object
user terminal
display
display control
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2023/025317
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
志織 安江
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NTT Docomo Inc
Original Assignee
NTT Docomo Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NTT Docomo Inc filed Critical NTT Docomo Inc
Priority to JP2024538884A priority Critical patent/JP7744523B2/ja
Publication of WO2024029275A1 publication Critical patent/WO2024029275A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics

Definitions

  • One aspect of the present invention relates to a display control system that controls the display of virtual objects.
  • Patent Document 1 discloses a mechanism for providing augmented reality or mixed reality to a user by displaying a virtual object superimposed on a real space image displayed on a display device.
  • one aspect of the present invention is to provide a display control system that can maintain the quality of the user's virtual experience while shortening the time it takes for a virtual object to be displayed on a user terminal. do.
  • a display control system is a display control system that performs display control of a virtual object displayed on a display unit of a user terminal, and the display control system controls the display of a first virtual object, which is a first virtual object, in a first file.
  • a file storage unit that stores a first file having a file size and a second file having a second file size larger than the first file size; and a display control unit that controls display of the first virtual object on a user terminal.
  • the display control unit performs a first process of displaying a first virtual object based on the first file on the display unit by transmitting the first file to the user terminal with priority over the second file, and transmitting the first file.
  • a first file having a small file size (first file size) is preferentially transmitted to the user terminal.
  • first file size a first file having a small file size
  • the second file is transmitted to the user terminal.
  • the first virtual object based on the first file displayed on the display unit is switched to the first virtual object based on the second file.
  • the first virtual object based on the second file which has a larger file size and higher quality than the first file, can be presented to the user later, so that the quality of the user's virtual experience can be maintained.
  • a display control system that can maintain the quality of a user's virtual experience while reducing the time it takes for a virtual object to be displayed on a user terminal.
  • FIG. 1 is a diagram illustrating an example of a functional configuration of a system including a server and a user terminal, which is a display control system according to an embodiment.
  • FIG. 3 is a diagram showing an example of file generation processing.
  • FIG. 3 is a diagram schematically showing an example of display control of a virtual object.
  • FIG. 2 is a sequence diagram illustrating an example of operations of a server and a user terminal. It is a diagram showing an example of the hardware configuration of a server and a user terminal.
  • FIG. 1 is a diagram showing an example of a system including a server 10, which is a display control system according to an embodiment, and a user terminal 20.
  • the server 10 is a device that controls the display of virtual objects displayed on the display unit 24 of the user terminal 20.
  • the server 10 provides the user with an AR experience or an MR experience by displaying a virtual object together with a real space image on the display unit 24 of the user terminal 20.
  • the server 10 may provide the user with a VR experience by displaying a virtual object together with a virtual background (virtual space image) on the display unit 24 of the user terminal 20.
  • the user terminal 20 is a computer terminal used by a user. The type of user terminal 20 is not limited.
  • the user terminal 20 may be a personal computer such as a desktop PC or a laptop PC, or may be a terminal device such as a smartphone or tablet held in the user's hand, or may be a terminal device such as a user's head or the like. It may also be a wearable device that is worn on a part of the body. Examples of wearable devices include goggle-type, glass-type (glasses-type), hat-type, and other head-mounted displays.
  • the server 10 and the user terminal 20 are connected to each other so as to be able to communicate data via a wired or wireless communication network.
  • the server 10 and the user terminal 20 may be connected via one or more other devices.
  • each of the server 10 and the user terminal 20 may be configured to be able to communicate with other devices such as the user's smartphone.
  • the server 10 is configured to be able to communicate with the user terminal 20 via the other device.
  • the server 10 includes a reception section 11 (terminal information acquisition section), a file generation section 12, a file storage section 13, and a display control section 14. Further, the user terminal 20 includes an imaging section 21, a transmitting section 22, a receiving section 23, and a display section 24.
  • the imaging unit 21 has a function of capturing an image of the real space around (for example, in front of) the user terminal 20 to obtain a real space image (imaging data).
  • the imaging unit 21 is configured by, for example, a camera.
  • the real space image may be a still image or a moving image (video).
  • the real space image may be displayed on the display unit 24.
  • the imaging data is a plurality of images obtained by capturing images of an object (real object) existing in real space from a plurality of different directions. It may also be composed of image data.
  • the transmitting unit 22 transmits the imaging data acquired by the imaging unit 21 to the server 10. Further, in this embodiment, the transmitter 22 transmits terminal information regarding the attributes of the user terminal 20 to the server 10. Examples of attributes of the user terminal 20 include application type, communication speed, processing performance, and the like.
  • Application type is information indicating the type of application used to display the virtual object sent from the server 10.
  • the application type can be specified, for example, by a combination of the type of user terminal 20 and the type of application running on the user terminal 20.
  • Examples of application types include a dedicated application that runs on a PC, a web browser that runs on a PC, a web browser that runs on a smartphone, and an application built into a head-mounted display such as smart glasses.
  • the transmitter 22 transmits terminal information indicating the application type to the server 10, for example, when starting execution of a service that displays a virtual object transmitted from the server 10 (for example, during a login operation).
  • Communication speed is the download speed (download speed) when downloading data from the server 10 to the user terminal 20.
  • the user terminal 20 is a terminal that moves with the user (e.g., a head-mounted display, a smartphone, etc.) and the server 10 and the user terminal 20 communicate wirelessly, the strength of the radio waves changes depending on the location of the user terminal 20. Therefore, the communication speed may change.
  • the transmitter 22 periodically transmits terminal information indicating the communication speed to the server 10.
  • Processing performance is information indicating the specifications of the processor, memory, etc. that the user terminal 20 has, the processor usage rate, the memory usage rate, etc.
  • terminal information indicating processing performance at a certain point in time is information indicating the amount of computational resources (processor, memory, etc.) available on the user terminal 20 at that point in time.
  • the transmitter 22 periodically transmits terminal information indicating processing performance to the server 10.
  • the receiving unit 23 receives information for displaying a virtual object on the display unit 24 (a first file, a second file, etc. to be described later) from the server 10.
  • the display unit 24 displays a virtual object in a display area 24a provided in the display unit 24 based on instructions from the display control unit 14 of the server 10. Specifically, the display unit 24 displays the virtual object in the display area 24a based on information for displaying the virtual object (first file, second file, etc.) received from the server 10.
  • the display unit 24 may be configured by a display monitor.
  • the display section 24 may be configured with a touch panel display.
  • the display unit 24 may be configured by a display placed in front of both eyes of the user.
  • the receiving unit 11 receives various data transmitted by the user terminal 20 (transmitting unit 22). In this embodiment, the receiving unit 11 receives imaging data acquired by the imaging unit 21.
  • the receiving unit 11 also functions as a terminal information acquisition unit that acquires the above-mentioned terminal information (application type, communication speed, processing performance, etc.).
  • the file generation unit 12 acquires the imaging data received by the reception unit 11.
  • the imaging data includes a real object that is an object existing in real space
  • the file generation unit 12 generates a first file and a second file of a virtual object corresponding to the real object based on the imaging data. do.
  • the file generation unit 12 corresponds to a real object included in the imaging data as a subject, for example, by performing at least one of image recognition (2D image recognition), photogrammetry, and 3D scanning on the imaging data. Generate a file representing a virtual object.
  • the method of generating a virtual object by the file generation unit 12 is not limited to the generation method based on real imaging data as described above.
  • the file generation unit 12 may generate a file representing a virtual object by modeling or the like. That is, the virtual objects handled in this embodiment are not limited to objects based on real objects, but may be objects created by creators such as CG designers.
  • FIG. 2 shows an example in which a rabbit kept as a pet by a user is imaged as a real object by the user terminal 20 (imaging unit 21).
  • the file generation unit 12 generates three types of files 50A, 50B, and 50C as data representing a virtual object (first virtual object) corresponding to this real object.
  • the files 50A, 50B, and 50C are all data used to display a virtual object on the display unit 24 of the user terminal 20, and may include drawing information for drawing the virtual object in the display area 24a.
  • the virtual object is expressed as a 3D object.
  • the file 50A is data that has not been compressed (reduced) and retains its original quality (resolution). That is, the file 50A is an original file generated based on imaging data of a real object.
  • the file 50B is data obtained by compressing the file 50A by 30%. That is, the file size (file capacity) of the file 50B is 70% of the file size of the file 50A.
  • the file 50C is data obtained by compressing the file 50A by 60%. That is, the file size of the file 50C is 40% of the file size of the file 50A.
  • Files 50B and 50C have lower quality than file 50A due to the data compression described above.
  • the surface shape of the virtual object (rabbit) represented by the file 50B is rougher than the surface shape of the virtual object represented by the original file 50A.
  • the surface shape of the virtual object represented by the file 50C is rougher than the surface shape of the virtual object represented by the file 50B. That is, the files 50B and 50C have a lower quality (resolution) than the original file 50A by the extent that the file size is smaller than the original file size.
  • the file generation unit 12 generates two types of compressed files (files 50B and 50C), but the file generation unit 12 generates only one type of compressed file (one specific compression rate).
  • three or more types (three or more different compression ratios) of compressed files may be generated.
  • the file generation unit 12 may determine which compression ratio to generate a compressed file according to the file size of the original file 50A. For example, if the file size of the original file 50A is not very large, the file size can be made sufficiently small without increasing the compression ratio that much. In this case, the file generation unit 12 may generate only the file 50B.
  • the file storage unit 13 acquires and stores, for a first virtual object, a first file having a first file size and a second file having a second file size larger than the first file size.
  • the first virtual object is a virtual object corresponding to the real object (rabbit) shown in FIG. 2. That is, in the present embodiment, the first virtual object is a virtual object generated based on imaging data of the real object.
  • file 50C corresponds to a first file having a first file size.
  • each of the files 50A and 50B corresponds to a second file having a second file size larger than the first file size. That is, in the example of FIG. 2, the file storage unit 13 stores a plurality (here, two) of second files (files 50A and 50B) having mutually different file sizes. Note that when the file generation unit 12 generates only the files 50A and 50B, the file 50B functions as the first file and the file 50A functions as the second file.
  • the file storage unit 13 stores the files 50A, 50B (second file) and file 50C (first file) generated by the file generation unit 12 in association with the virtual object corresponding to the real object. do. That is, the file storage unit 13 stores (registers) a first file and a second file corresponding to each virtual object.
  • the file storage unit 13 manages a plurality of virtual objects (that is, a plurality of virtual objects each corresponding to a different real object) using, for example, unique identification information (object ID). For example, when a virtual object is operated on the web (that is, when displayed by a web browser of the user terminal 20), the file storage unit 13 stores a unique URI (Uniform Resource Identifier) for a file representing each virtual object. ) etc. may be assigned.
  • object ID unique identification information
  • first file and second file in this embodiment, three files 50A, 50B, and 50C regarding the same virtual object can be managed as follows.
  • files 50A, 50B, and 50C are managed by file name.
  • the file storage unit 13 names the file 50A (original file) as “usagi.3d”, the file 50B (the file to which 30% compression has been applied) as “usagi_a.3d”, and the file 50C. (The file to which 60% compression has been applied) is managed (stored) as the file name "usagi_b.3d”.
  • "usagi” is character string information that uniquely identifies a virtual object (here, as an example, a virtual object corresponding to the real object (rabbit) shown in FIG. 2).
  • ".3d” is an example of an identifier given to each file (data).
  • “_a” and “_b” connected after “usagi” are information corresponding to the compression rate for the original file. According to the first example, it is possible to understand the virtual object and the compression rate based on the file name.
  • the files 50A, 50B, and 50C are managed by a method of embedding information regarding the compression rate (the compression rate of the original file is 0) in the header portion of each file.
  • the file storage unit 13 saves the files 50A, 50B, and 50C in different locations, and gives the files 50A, 50B, and 50C a common file name (for example, "usagi.3d"). good.
  • multiple files 50A, 50B, and 50C corresponding to the same virtual object are managed by a management table.
  • multiple files 50A, 50B, and 50C corresponding to the same virtual object are associated with each other in the management table by a common key indicating the virtual object.
  • the file management method is not limited to the first to third examples above, and may be managed using methods other than these.
  • the display control unit 14 controls the display of the first virtual object on the user terminal 20. For example, when it becomes necessary to display a first virtual object on the display unit 24 of the user terminal 20, the display control unit 14 displays files (first file, second file) corresponding to the first virtual object on the user terminal 20.
  • the display control described above is executed by controlling the processing provided to the. Specifically, the display control unit 14 executes a first process, a second process, and a third process described below. An example of processing by the display control unit 14 will be described below with reference to FIG. 3.
  • the first process transmits the first file (file 50C in the example of FIG. 2) to the user terminal 20 with priority over the second file (files 50A and 50B in the example of FIG. 24 to display the first virtual object based on the first file. That is, the display control unit 14 gives priority to the file 50C whose file size has been reduced over the original file and provides it to the user terminal 20.
  • the first virtual object based on the file 50C is The time required to display the file in the area 24a) can be made shorter than when the original file (file 50A) is sent to the user terminal 20.
  • the second file (file 50A or file 50B) is transmitted to the user terminal 20.
  • This is a process of transmitting to the terminal 20.
  • the display control unit 14 uses a first control pattern to start sending the second file to the user terminal 20 after sending the first file, and a first control pattern to start sending the second file to the user terminal 20 after sending the second file to the user terminal 20.
  • the second control pattern may be executed in parallel with the transmission of the first file to.
  • the transmission process of the first file is completed before the transmission process of the second file also in the second control pattern.
  • the transmission process of the second file is executed in parallel with the transmission process of the first file, so the point in time when the transmission of the second file is completed in the second control pattern is the same as that in the first control pattern.
  • This is a time before the time when the transmission of the second file is completed. That is, the second control pattern starts the transmission process of the second file before the transmission of the first file is completed, and the first control pattern starts the transmission process of the second file after the transmission of the first file is completed.
  • the configuration is such that transmission of the second file can be completed earlier than the control pattern.
  • the time point at which transmission of the first file is completed in the second control pattern is later than the time point at which transmission of the first file is completed in the first control pattern.
  • the second file is also transmitted while the first file is being transmitted (i.e., communication resources are allocated between the first file and the second file). 2 files), the time required to transmit the first file is longer than that of the first control pattern.
  • the first control that sequentially executes the download of the first file and the download of the second file is recommended.
  • pattern is more advantageous than the second control pattern.
  • the first The second control pattern which performs parallel downloading of the file and the second file, is more advantageous than the first control pattern.
  • the display control unit 14 may be configured to be able to switch between the first control pattern and the second control pattern. For example, the display control unit 14 may determine which of the first control pattern and the second control pattern to execute based on user settings. That is, the display control unit 14 may allow the user to select which of the first control pattern and the second control pattern to execute. Alternatively, the display control unit 14 may determine which of the first control pattern and the second control pattern to execute, depending on the above-mentioned terminal information (for example, communication speed).
  • the display control unit 14 may determine which of the first control pattern and the second control pattern to execute, depending on the above-mentioned terminal information (for example, communication speed).
  • the display control unit 14 may determine which of the first control pattern and the second control pattern to execute, depending on the communication method installed in the user terminal 20. According to the above configuration, each of the above-described first control pattern and second control pattern can be appropriately switched according to the use case.
  • the display control unit 14 selects a second file to be transmitted to the user terminal 20 from among the plurality of second files (files 50A and 50B in this embodiment) based on the terminal information. may be determined. For example, the display control unit 14 may determine the second file to be provided to the user terminal 20 according to the above-mentioned attributes of the user terminal 20 (application type, communication speed, processing performance, etc.).
  • the display control unit 14 selects a quality level that does not exceed the maximum quality that can perform display processing on the user terminal 20, based on terminal information that includes information regarding at least one of the application type and processing performance of the user terminal 20. 2 file may be determined as the second file to be provided to the user terminal 20.
  • the display control unit 14 may select the file 50B as the second file to be transmitted to the user terminal 20.
  • the display control unit 14 transmits a second file that is expected to be downloaded to the user terminal 20 within a predetermined threshold time based on terminal information including information regarding the communication speed of the user terminal 20. It may also be determined as the second file to be provided. For example, in the example of FIG.
  • the display control The unit 14 may select the file 50B as the second file to be transmitted to the user terminal 20.
  • the user terminal Based on the attributes 20, it becomes possible to provide (transmit) a second file with appropriate quality to the user terminal 20.
  • the file 50B is determined as the second file to be transmitted to the user terminal 20 through the above-described determination process.
  • the first virtual object based on the first file (file 50C) is displayed on the display unit 24 (display area 24a), and the second file (here, as an example, the file 50B) is completed, the first virtual object based on the file 50C displayed on the display unit 24 is switched to the first virtual object based on the file 50B (see arrow P1 in FIG. 3). .
  • an identifier specified as a display target on the user terminal 20 (here, an identifier corresponding to file 50C (for example, the above-mentioned URI, object ID, etc.)) is transferred to file 50B. This can be done by changing the identifier to the corresponding identifier. Furthermore, information such as the angle of view, position, distance (depth), etc. of the first virtual object based on the file 50C that was displayed on the display unit 24 immediately before switching may be configured to be carried over even after switching. In this case, it is possible to smoothly switch from the file 50C to the file 50B while maintaining the way the first virtual object is viewed by the user.
  • the display control unit 14 may determine whether or not a processing delay of the user terminal 20 has occurred based on information including information regarding the processing performance of the user terminal 20 (for example, the above-mentioned terminal information). .
  • the display control unit 14 may determine that a processing delay has occurred when the processor usage rate of the user terminal 20 is equal to or higher than a threshold value.
  • the display control unit 14 determines that a processing delay has occurred, the display control unit 14 allows the user to switch back the process (see arrow P2 in FIG. 3) from the second file (file 50B) to the first file (file 50C).
  • the terminal 20 may be executed. Since the file size of the file 50C is smaller than that of the file 50B, the amount of calculation required for display processing (drawing processing) and the amount of memory used are smaller than those of the file 50B. Therefore, according to the display control described above, by executing the cutback process in a situation where a processing delay has occurred, it is possible to eliminate the processing delay.
  • the display control unit 14 also controls the display unit 24 (display area 24a) to display the first virtual object based on the second file (file 50B), and the display unit 24 to display a plurality of virtual objects including the first virtual object.
  • the first virtual object based on the file 50B displayed on the display section 24 is transferred to the first file (file 50C) according to the number of virtual objects displayed on the display section 24.
  • Display control may be performed to switch back to the first virtual object based on the first virtual object.
  • the display control unit 14 acquires information indicating the number of virtual objects displayed on the display unit 24.
  • the display control unit 14 may acquire such information from the user terminal 20, or the display control unit 14 may acquire the virtual object displayed on the display unit 24 based on the history of sending virtual object files to the user terminal 20. You may also know the number of Then, for example, when the display control unit 14 determines that the number of displayed virtual objects is greater than or equal to a predetermined threshold, the display control unit 14 switches the second file (file 50B) to the first file (file 50C). You may cause the user terminal 20 to execute the return process (see arrow P2 in FIG. 3). According to the display control described above, processing delays can be avoided by executing cutback processing in situations where there is a high possibility that processing delays have occurred (or are likely to occur in the future) due to the large number of displayed virtual objects. It is possible to eliminate or avoid this problem.
  • step S1 the imaging unit 21 images a real object.
  • step S2 the transmitter 22 transmits the imaging data acquired in step S1 to the server 10.
  • the transmitter 22 also transmits terminal information (in this embodiment, information including application type, communication speed, processing performance, etc., as an example) to the server 10.
  • the imaging data and terminal information are received by the receiving unit 11 of the server 10.
  • step S3 the file generation unit 12 generates a file 50A (second file), a file 50B (second file), and a file 50C (first file) based on the imaging data (see FIG. 2).
  • step S4 the file storage section 13 stores the files 50A, 50B, and 50C generated by the file generation section 12.
  • step S5 the display control unit 14 sends the first file (file 50C in the example of FIG. 3) to the user terminal 20 with priority over the second file (file 50B in the example of FIG. 3).
  • step S6 the first virtual object based on the first file (file 50C) is displayed on the display unit 24 (see the upper part of FIG. 3).
  • step S7 the display control unit 14 executes a second process. That is, the display control unit 14 transmits the second file (file 50B) to the user terminal 20 after transmitting the first file (file 50C) or in parallel with transmitting the first file. Since the file size of the second file (second file size) is larger than the file size of the first file (first file size), the second file is sent to the user terminal 20 after step S6. That is, the time point at which the reception of the second file is completed at the user terminal 20 is a time point after step S6.
  • step S8 the display control unit 14 executes a third process. That is, the display control unit 14 displays the first virtual object based on the first file (file 50C) on the display unit 24 (display area 24a), and displays the second file (file 50B) to the user terminal 20. , the first virtual object based on the file 50C displayed on the display unit 24 is switched to the first virtual object based on the file 50B (see arrow P1 in FIG. 3). For example, the display control unit 14 transmits to the user terminal 20 together with the second file instruction information that instructs to "switch the first file to the second file after receiving the second file". can perform the switching process as described above.
  • step S9 the display control unit 14 performs display control to switch back the first virtual object based on the file 50B displayed on the display unit 24 to the first virtual object based on the first file (file 50C). Determine whether or not.
  • This determination process is performed periodically or irregularly based on terminal information (for example, information regarding processing performance) received from the user terminal 20, information indicating the number of virtual objects displayed on the display unit 24, etc. obtain.
  • the display control unit 14 determines that a processing delay has occurred in the user terminal 20, or that the number of virtual objects displayed on the display unit 24 is greater than or equal to a threshold value. If so, the user terminal 20 is instructed to execute the switchback process (step S10). As a result, in step S11, a process of switching back from the second file (file 50B) to the first file (file 50C) is executed (see arrow P2 in FIG. 3).
  • step S8 the display control unit 14 determines the reason for the cutback (for example, that the above processing delay has occurred, that the number of displayed virtual objects is If it is determined that the problem (e.g., the current value is equal to or greater than a threshold value) has been resolved, the process of step S8 may be executed again.
  • the reason for the cutback for example, that the above processing delay has occurred, that the number of displayed virtual objects is If it is determined that the problem (e.g., the current value is equal to or greater than a threshold value) has been resolved, the process of step S8 may be executed again.
  • the first file (file 50C) having a small file size (first file size) is preferentially transmitted to the user terminal 20.
  • the waiting time until the first virtual object is displayed on the display unit 24 of the user terminal 20 can be shortened.
  • the second file (file 50B) is transmitted to the user terminal 20.
  • the first virtual object based on the first file displayed on the display unit 24 is switched to the first virtual object based on the second file (arrow in FIG. 3). (See P1).
  • the first virtual object based on the second file which has a larger file size and higher quality than the first file, can be presented to the user later, so that the quality of the user's virtual experience can be maintained.
  • the file generation unit 12 generates the first file and the second file of the virtual object corresponding to the real object based on the imaging data
  • the file storage unit 13 generates the first file and the second file of the virtual object corresponding to the real object based on the imaging data.
  • the first file and the second file are stored in association with the virtual object.
  • the display control system was configured by the server 10, but the display control system may be configured by the server 10 alone, or by the server 10 and the user terminal 20. That is, some of the functions of the server 10 described above may be executed by the user terminal 20.
  • the display control unit 14 of the server 10 controls the display content of the display unit 24 of the user terminal 20, but the display control unit 14 may be provided in the user terminal 20 instead of the server 10. Further, some of the functions of the server 10 described above may be omitted.
  • the server 10 file generation unit 12
  • the virtual object file may be generated by a device different from the server 10.
  • the display control system of the present disclosure (server 10 in the above embodiment) has the following configuration.
  • a display control system that controls the display of virtual objects displayed on a display unit of a user terminal, a file storage unit that stores a first file having a first file size and a second file having a second file size larger than the first file size for a first virtual object that is one of the virtual objects; , a display control unit that controls display of the first virtual object on the user terminal,
  • the display control section includes: a first process of displaying the first virtual object based on the first file on the display unit by transmitting the first file to the user terminal with priority over the second file; a second process of transmitting the second file to the user terminal after transmitting the first file or in parallel with transmitting the first file; Displayed on the display unit when the first virtual object based on the first file is displayed on the display unit and transmission of the second file to the user terminal is completed. performing a third process of switching the first virtual object based on the first file to the first virtual object based on the second file; Display control system.
  • [2] Obtain imaging data obtained by imaging a real object that exists in real space, and create the first file and the second file of the virtual object corresponding to the real object based on the imaging data. further comprising a file generation unit that generates a file, The file storage unit stores the first file and the second file generated by the file generation unit in association with the virtual object corresponding to the real object.
  • the display control system according to [1].
  • the display control unit controls the display unit to display the first virtual object on the display unit depending on the occurrence of processing delay of the user terminal. cutting back the first virtual object based on the second file that has been updated to the first virtual object based on the first file;
  • the display control system according to [1] or [2].
  • the display control unit displays the first virtual object based on the second file on the display unit, and displays the plurality of virtual objects including the first virtual object on the display unit. , the first virtual object based on the second file displayed on the display unit is changed to the first virtual object based on the first file according to the number of virtual objects displayed on the display unit. Cut back to the first virtual object, The display control system according to any one of [1] to [3].
  • the display control unit includes a first control pattern for starting transmission of the second file to the user terminal after transmission of the first file, and a first control pattern for starting transmission of the second file to the user terminal after transmitting the first file. a second control pattern in which file transmission is performed in parallel with transmission of the first file; and a second control pattern configured to be switchable.
  • the display control system according to any one of [1] to [4].
  • [6] Further comprising a terminal information acquisition unit that acquires terminal information indicating attributes of the user terminal,
  • the file storage unit stores a plurality of second files having mutually different file sizes,
  • the display control unit determines the second file to be transmitted to the user terminal from among the plurality of second files, based on the terminal information.
  • the display control system according to any one of [1] to [5].
  • each functional block may be realized using one physically or logically coupled device, or may be realized using two or more physically or logically separated devices directly or indirectly (e.g. , wired, wireless, etc.) and may be realized using a plurality of these devices.
  • the functional block may be realized by combining software with the one device or the plurality of devices.
  • Functions include judgment, decision, judgment, calculation, calculation, processing, derivation, investigation, exploration, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, consideration, These include, but are not limited to, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating, mapping, and assigning. I can't.
  • the server 10 in an embodiment of the present disclosure may function as a computer that performs the display control method of the present disclosure.
  • FIG. 5 is a diagram illustrating an example of the hardware configuration of the server 10 according to an embodiment of the present disclosure.
  • the server 10 may be physically configured as a computer device including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, and the like.
  • the user terminal 20 may also have the same hardware configuration as the server 10.
  • the word “apparatus” can be read as a circuit, a device, a unit, etc.
  • the hardware configuration of the server 10 may include one or more of the devices shown in FIG. 5, or may not include some of the devices.
  • Each function in the server 10 is performed by loading predetermined software (programs) onto hardware such as the processor 1001 and the memory 1002, so that the processor 1001 performs calculations, controls communication by the communication device 1004, and controls the memory 1002 and the memory 1002. This is realized by controlling at least one of reading and writing data in the storage 1003.
  • the processor 1001 for example, operates an operating system to control the entire computer.
  • the processor 1001 may be configured by a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic unit, registers, and the like.
  • CPU central processing unit
  • the processor 1001 reads programs (program codes), software modules, data, etc. from at least one of the storage 1003 and the communication device 1004 to the memory 1002, and executes various processes in accordance with these.
  • programs program codes
  • software modules software modules
  • data etc.
  • the program a program that causes a computer to execute at least part of the operations described in the above embodiments is used.
  • each functional unit of the server 10 for example, the display control unit 14, etc.
  • each functional unit of the server 10 for example, the display control unit 14, etc.
  • Processor 1001 may be implemented by one or more chips. Note that the program may be transmitted from a network via a telecommunications line.
  • the memory 1002 is a computer-readable recording medium, and includes at least one of ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), etc. may be done.
  • Memory 1002 may be called a register, cache, main memory, or the like.
  • the memory 1002 can store executable programs (program codes), software modules, and the like to implement a display control method according to an embodiment of the present disclosure.
  • the storage 1003 is a computer-readable recording medium, such as an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, or a magneto-optical disk (for example, a compact disk, a digital versatile disk, or a Blu-ray disk). (registered trademark disk), smart card, flash memory (eg, card, stick, key drive), floppy disk, magnetic strip, etc.
  • Storage 1003 may also be called an auxiliary storage device.
  • the storage medium mentioned above may be, for example, a database including at least one of memory 1002 and storage 1003, a server, or other suitable medium.
  • the communication device 1004 is hardware (transmission/reception device) for communicating between computers via at least one of a wired network and a wireless network, and is also referred to as a network device, network controller, network card, communication module, etc., for example.
  • the input device 1005 is an input device (eg, keyboard, mouse, microphone, switch, button, sensor, etc.) that accepts input from the outside.
  • the output device 1006 is an output device (for example, a display, a speaker, an LED lamp, etc.) that performs output to the outside. Note that the input device 1005 and the output device 1006 may have an integrated configuration (for example, a touch panel).
  • each device such as the processor 1001 and the memory 1002 is connected by a bus 1007 for communicating information.
  • the bus 1007 may be configured using a single bus, or may be configured using different buses for each device.
  • the server 10 also includes hardware such as a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), and a field programmable gate array (FPGA).
  • DSP digital signal processor
  • ASIC application specific integrated circuit
  • PLD programmable logic device
  • FPGA field programmable gate array
  • the input/output information may be stored in a specific location (for example, memory) or may be managed using a management table. Information etc. to be input/output may be overwritten, updated, or additionally written. The output information etc. may be deleted. The input information etc. may be transmitted to other devices.
  • Judgment may be made using a value expressed by 1 bit (0 or 1), a truth value (Boolean: true or false), or a comparison of numerical values (for example, a predetermined value). (comparison with a value).
  • notification of prescribed information is not limited to being done explicitly, but may also be done implicitly (for example, not notifying the prescribed information). Good too.
  • Software includes instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, whether referred to as software, firmware, middleware, microcode, hardware description language, or by any other name. , should be broadly construed to mean an application, software application, software package, routine, subroutine, object, executable, thread of execution, procedure, function, etc.
  • software, instructions, information, etc. may be sent and received via a transmission medium.
  • a transmission medium For example, if the software uses wired technology (coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), etc.) and/or wireless technology (infrared, microwave, etc.) to create a website, When transmitted from a server or other remote source, these wired and/or wireless technologies are included within the definition of transmission medium.
  • wired technology coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), etc.
  • wireless technology infrared, microwave, etc.
  • data, instructions, commands, information, signals, bits, symbols, chips, etc. which may be referred to throughout the above description, may refer to voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, light fields or photons, or any of these. It may also be represented by a combination of
  • information, parameters, etc. described in this disclosure may be expressed using absolute values, relative values from a predetermined value, or using other corresponding information. may be expressed.
  • the phrase “based on” does not mean “based solely on” unless explicitly stated otherwise. In other words, the phrase “based on” means both “based only on” and “based at least on.”
  • any reference to elements using the designations "first,” “second,” etc. does not generally limit the amount or order of those elements. These designations may be used in this disclosure as a convenient way to distinguish between two or more elements. Thus, reference to a first and second element does not imply that only two elements may be employed or that the first element must precede the second element in any way.
  • a and B are different may mean “A and B are different from each other.” Note that the term may also mean that "A and B are each different from C”. Terms such as “separate” and “coupled” may also be interpreted similarly to “different.”
  • 10 Server (display control system), 11... Receiving unit, 12... File generating unit, 13... File storage unit, 14... Display control unit, 20... User terminal, 24... Display unit, 24a... Display area, 50A... File (second file), 50B... file (second file), 50C... file (first file).

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Information Transfer Between Computers (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
PCT/JP2023/025317 2022-08-02 2023-07-07 表示制御システム Ceased WO2024029275A1 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2024538884A JP7744523B2 (ja) 2022-08-02 2023-07-07 表示制御システム

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022-123279 2022-08-02
JP2022123279 2022-08-02

Publications (1)

Publication Number Publication Date
WO2024029275A1 true WO2024029275A1 (ja) 2024-02-08

Family

ID=89849227

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/025317 Ceased WO2024029275A1 (ja) 2022-08-02 2023-07-07 表示制御システム

Country Status (2)

Country Link
JP (1) JP7744523B2 (https=)
WO (1) WO2024029275A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7827282B1 (ja) * 2025-06-23 2026-03-10 クラスター株式会社 情報処理方法、情報処理装置、情報処理プログラム、及び情報処理システム

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11144089A (ja) * 1997-11-11 1999-05-28 Hitachi Ltd グラフィック処理装置及び方法
JP2002279449A (ja) * 2001-03-19 2002-09-27 Mitsubishi Electric Corp ３次元空間データ送信表示装置、３次元空間データ送信方法、３次元空間データ送信方法をコンピュータに実行させるためのプログラムを記録したコンピュータ読み取り可能な記録媒体
JP2004213641A (ja) * 2002-12-20 2004-07-29 Sony Computer Entertainment Inc 画像処理装置、画像処理方法、情報処理装置、情報処理システム、半導体デバイス、コンピュータプログラム
JP2019054488A (ja) * 2017-09-19 2019-04-04 キヤノン株式会社 提供装置および提供方法、プログラム

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11144089A (ja) * 1997-11-11 1999-05-28 Hitachi Ltd グラフィック処理装置及び方法
JP2002279449A (ja) * 2001-03-19 2002-09-27 Mitsubishi Electric Corp ３次元空間データ送信表示装置、３次元空間データ送信方法、３次元空間データ送信方法をコンピュータに実行させるためのプログラムを記録したコンピュータ読み取り可能な記録媒体
JP2004213641A (ja) * 2002-12-20 2004-07-29 Sony Computer Entertainment Inc 画像処理装置、画像処理方法、情報処理装置、情報処理システム、半導体デバイス、コンピュータプログラム
JP2019054488A (ja) * 2017-09-19 2019-04-04 キヤノン株式会社 提供装置および提供方法、プログラム

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7827282B1 (ja) * 2025-06-23 2026-03-10 クラスター株式会社 情報処理方法、情報処理装置、情報処理プログラム、及び情報処理システム

Also Published As

Publication number Publication date
JPWO2024029275A1 (https=) 2024-02-08
JP7744523B2 (ja) 2025-09-25

Similar Documents

Publication Publication Date Title
US10176549B2 (en) System, method and computer program product for remote graphics processing
US10354430B2 (en) Image update method, system, and apparatus
JP6475256B2 (ja) コンピュータ、制御デバイス及びデータ処理方法
US10397627B2 (en) Desktop-cloud-based media control method and device
US7853672B2 (en) Server device, control instruction processing method therefor, and terminal device
JP6276408B2 (ja) 画像表示方法および装置
WO2024029275A1 (ja) 表示制御システム
WO2019062175A1 (zh) 信息输入方法及装置
CN114820272B (zh) 数据交互的方法、装置、存储介质及电子设备
CN113743288A (zh) 一种云手机的图像识别方法、装置、设备及存储介质
EP3767464A1 (en) Information processing system, information processing apparatus, information processing method, and program
JP7733815B2 (ja) 仮想空間提供装置
CN111770165A (zh) 用于上传文件的方法、装置、电子设备及可读存储介质
CN105727556A (zh) 一种图像绘制的方法、相关设备及系统
CN114489572B (zh) 业务需求调整方法和相关设备
CN116643698B (zh) 数据写入方法及装置、电子设备和存储介质
CN113589927A (zh) 分屏显示方法、头戴式显示设备和计算机可读介质
US20260017004A1 (en) Rendering method, electronic device and non-transitory computer-readable storage medium
WO2024029190A1 (ja) 仮想オブジェクト生成装置
JP7608622B2 (ja) 表示制御装置
US20220308819A1 (en) Method for processing data and electronic device therefor
WO2024029497A1 (ja) 仮想空間提供システム
WO2022190735A1 (ja) 表示制御装置
JP2025122937A (ja) 対話処理システムおよび対話処理方法
JP2024075801A (ja) 表示制御装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23849834

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024538884

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 23849834

Country of ref document: EP

Kind code of ref document: A1