KR102141596B1 - Virtual reality player and integrated management system for monitoring thereof - Google Patents

Abstract

The present disclosure relates to a playback device for virtual reality content, a method for monitoring the same, and an integrated management system. The playback device includes a content analysis unit for analyzing virtual reality content and extracting image data; A format detection unit that detects the format of the image data and determines whether it can be supported by the playback device; An image processing unit for processing image data based on the detected format and outputting it as a frame-by-frame image when it is determined that the reproduction apparatus can support the image; It may include a display unit for reproducing the image-processed image data in units of frames. The playback device may further include a frame management unit to calculate the current frame number of the video data by counting the number of played frames, and in particular, may support ASTC format video data.

Description

VR playback device and integrated management system for monitoring it {VIRTUAL REALITY PLAYER AND INTEGRATED MANAGEMENT SYSTEM FOR MONITORING THEREOF}

The present disclosure relates to a playback device for virtual reality content, a method for monitoring the same, and an integrated management system. More specifically, a universal VR playback device capable of supporting ASTC format video data, and a method and an integrated management system that can efficiently monitor and control multiple VR playback devices by receiving the current frame number from the VR playback device during playback To explain.

Unless otherwise stated herein, the content described in this section is not prior art to the claims in this application and should not be construed as prior art for the reasons set forth in this section.

Virtual reality refers to an interface between a human and a computer that makes a specific environment or situation desired by a computer and makes a user feel as if they are interacting with the actual surrounding situation and environment. In order for the user to feel reality through virtual reality, information obtained through the user's five senses, such as vision, hearing, and tactile sense, must be provided. In particular, the most demanding information among the user's senses is vision, and devices similar to the eyes have been developed according to the development of virtual reality. The device developed as a result of these efforts is the head mounted display (HMD).

An HMD is a display device mounted on a user's head, and its types and principles vary. Initially, HMD was mainly used in the field of military equipment, but it is becoming popular as it is used in the game field. For example, current HMDs such as the Oculus Rift move the scene displayed on the screen to the left when the user rotates the face to the left, and the scene displayed on the screen moves to the right when the user rotates the face to the right. That is, the HMD itself recognizes the user's movement and reflects it on the screen. With HMD, it can be flexibly applied to suit a specific situation without requiring a lot of physical operating space. In the case of registered patent 10-1279869, a flight simulator image display apparatus and method using a head mounted display are disclosed. Specifically, in this document, a head mounted display, that is, an HMD is applied to a flight simulator, giving the user a sense of realism as if they were actually flying when operating the simulator.

Meanwhile, as the demand for HMD increases commercially, the demand for related virtual reality devices also increases. In particular, in order to provide the best virtual reality experience to users, the development of virtual reality devices associated with each virtual reality content has also increased. For example, Patent No. 10-1232422 discloses a motion simulator system for virtual skin scuba sensation, and a user can experience skin scuba using a virtual reality content and a motion simulator including an HMD and a headset. A motion simulator is a device that allows a user to feel movement in connection with virtual reality content, and a user can feel a sense of immersion in the virtual reality content by riding on a motion simulator while wearing an HMD.

Recently, a lot of VR theme parks have been created around the city center, and new VR devices have been introduced to allow users to experience virtual reality content in existing amusement parks and amusement rooms, and access to places where VR experiences can be increased. In order to enable multiple experiencers to experience virtual reality content in such a limited space, a plurality of mobile HMDs are used, and for convenience, the experiencer uses a smart phone terminal mounted on a VR device to use as a HMD. do. This mobile method has the advantage of being able to connect wirelessly, so there is no restriction on the experience space, and multiple experiencers can experience at the same time.

However, in the conventional management system for virtual reality content, a heating phenomenon occurs in the hardware of the VR device, and there is a problem that the mounted battery is quickly consumed. Since the virtual reality content is forcibly terminated when the internal temperature becomes too high or the battery reaches discharge due to the heat generation phenomenon, it is difficult for the user to stably experience the virtual reality content under the existing management system. Since event venues for experiencing virtual reality content such as VR theme parks are operated for a long time, the fever phenomenon and the low battery problem need to be solved in terms of the manager's operation and the safety of the experiencer.

In addition, in the conventional management system for virtual reality content, there is a problem in that it is impossible for an administrator to check and control the HMD state of an experienced person. In other words, even though VR theme park requires rapid operation of multiple HMDs by multiple concurrent users, it is difficult for managers to grasp the current video that the viewer is viewing. For this reason, the administrator has taken a method of asking and confirming whether or not the normal operation of each of a plurality of experienced users has been performed, so the time required to set up the VR device was long. Although the mirroring technology is used as a method for the administrator to grasp the video the viewer is viewing, in this case, there is a delay in capturing the streaming video from the VR device and sending it to the management server, so there is a limit to applying the mirroring technology. there was. Since the problem of the HMD status check and control impossible of the experience by the manager eventually leads to a decrease in operational convenience and stability, it is necessary to implement an integrated management system to enable this.

KR 10-1279869 B1 KR 10-1232422 B1

The present disclosure relates to a playback device for virtual reality content, a method for monitoring the same, and an integrated management system.

In one embodiment, a playback device for virtual reality content is described. The playback device includes a content analysis unit that analyzes virtual reality content stored in the playback device and extracts image data; A format detection unit that detects the format of the image data and determines whether it can be supported by the playback device; An image processing unit for processing image data based on the detected format and outputting it as a frame-by-frame image when it is determined that the reproduction apparatus can support the image; A display unit for reproducing image processed image data in units of frames; And a frame management unit counting the number of reproduced frames to calculate a current frame number of image data. The playback device may support image data in an adaptive scalable texture compression (ASTC) format.

The regeneration device may include a temperature sensing unit, and the temperature sensing unit may sense the heating temperature of the regeneration device. The playback device may include a usage checking unit, and the usage checking unit may check at least one of CPU usage, memory usage, and battery level of the playback device. The playback device may include a content search unit and a content selection unit. The content search unit may search for virtual reality content stored in the playback device. The content selection unit may display a playable virtual reality content according to a search result of the content search unit and provide a user interface for selecting virtual reality content to be played. The playback device may include an eye tracking unit, and the target tracking unit may recognize an eye of the playback device user and calculate an input value. The playback device may further include an audio processing unit that processes and outputs audio data, and the content analysis unit extracts audio data by analyzing virtual reality content, and the audio data is determined to have the same sync audio section as the frame unit of the video data. Can.

The playback device may further include a communication unit communicatively connected to the motion simulator. The content analysis unit analyzes the virtual reality content to extract motion data, and the communication unit transmits motion data to the motion simulator, and the motion data may have a synchro motion value equal to a frame unit of image data. The playback device may further include a communication unit communicatively connected to the integrated management device for controlling and monitoring the playback device. The communication unit may transmit at least one of a current frame number, fps (frame per second), heat generation temperature, CPU usage, memory usage, and battery remaining amount when the reproduction apparatus is played back to the integrated management device.

In another embodiment, an integrated management system is described. The integrated management system may include one or more playback devices for virtual reality content and an integrated management device that is communicatively connected to the playback devices and monitors each of them. The playback device analyzes the virtual reality content, extracts video data, detects the format of the video data, determines whether it is supported, and if it is determined that the playback device can support, processes the video data based on the detected format and frames unit The current frame number of the image data may be calculated by outputting the image, reproducing the image-processed image data in units of frames, and counting the number of reproduced frames. The integrated management device may monitor the playback status of each playback apparatus by receiving the current frame number of the video data being played back from each playback apparatus. The playback device may support ASTC format image data.

The regeneration device of the integrated management system calculates at least one of fps, heat generation temperature, CPU usage, memory usage, and battery level during regeneration, and the integrated management device fps, heat generation temperature, CPU usage, memory usage, and battery from each reproduction device The usage state of each device may be monitored by receiving at least one of the remaining amount. The integrated management device controls playback of one or more playback devices by transmitting an advertising signal to the playback device, exchanging master/slave data, or exchanging data using WIFI. can do.

The integrated management system may further include a motion simulator each communicatively connected to the playback device and the integrated management device. The motion simulator may receive motion data having a synchro motion value equal to a frame unit of image data from a playback device or an integrated management device, and provide a specific motion when playing each frame according to the motion value.

In another embodiment, a method for monitoring one or more playback devices for virtual reality content under the control of an integrated management server is described. A method for monitoring may include transmitting a playback signal to a playback device to cause the playback device to play virtual reality content on a frame-by-frame basis; Receiving a current frame number of the video data being played back from the playback device; And displaying an image corresponding to the current frame number.

The method for monitoring may further include transmitting motion data in which a motion value having a synchro motion value equal to a frame unit of image data is transmitted to a motion simulator simultaneously with transmission of a reproduction signal. A method for monitoring may include receiving, from a playback device, at least one of fps, heat generation temperature, CPU usage, memory usage, and battery level during playback as usage state information of the playback apparatus; Determining whether the usage status information is within a range of predetermined values for each; And when it is determined that one or more of the usage status information is not within a range of a predetermined value, generating a warning message.

In the method for monitoring, the step of transmitting a playback signal controls playback of one or more playback devices by transmitting an advertising signal to the playback device, exchanging master/slave data, or exchanging data using WIFI. It may include the steps. Receiving the current frame number may include receiving a sequence number of the ASTC sequence file. The receiving of the current frame number includes: calculating a maximum frame number among current frame numbers of the plurality of playback devices; And synchronizing the plurality of playback devices based on the maximum frame number.

The above brief summary and description of effects are merely illustrative and are not intended to limit the technical details intended in this disclosure. By referring to the following detailed description and accompanying drawings, in addition to the above-described exemplary embodiments and technical features, additional embodiments and technical features may be understood.

Features and other additional features of the present disclosure described above will be described in detail below with reference to the accompanying drawings. These drawings show only a few embodiments according to the present disclosure and should not be considered as limiting the scope of the technical spirit of the present disclosure. The technical spirit of the present disclosure will be described in more detail and in detail using the accompanying drawings.
1 is a schematic diagram of an integrated management system, according to at least some embodiments of the present disclosure.
2 is a diagram showing an example of the configuration of virtual reality content capable of playing cinematic VR.
3 is a diagram illustrating an example of a playback device for virtual reality content, according to at least some embodiments of the present disclosure.
4 is a diagram illustrating an example of a monitoring screen of an integrated management device, according to at least some embodiments of the present disclosure.
5 is a flow diagram illustrating an example process of a method for monitoring one or more playback devices for virtual reality content under the control of an integrated management server, in accordance with at least some embodiments of the present disclosure.

Hereinafter, embodiments and examples of the present application will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present disclosure pertains can easily implement them. However, the present application can be implemented in many different forms and is not limited to the embodiments and examples described herein.

The present disclosure relates to a playback device for virtual reality content, a method for monitoring the same, and an integrated management system. More specifically, by receiving the current frame number from a universal VR playback device capable of supporting ASTC format video data and a VR playback device during playback, it is possible to efficiently monitor and check the images of users experiencing multiple VR playback devices in real time. Describes an integrated management system that can be controlled.

As described above, the management system of the existing VR playback device has a problem that heat generation occurs in hardware, the battery is quickly consumed, and the administrator cannot check and control the HMD status of the experienced person. In the case of a hardware fever phenomenon, it is analyzed that the internal temperature rises due to excessive use of CPU resources when playing an MP4 (H.264) format VR image in an existing VR playback device. Therefore, it is an object of the present invention to develop a VR playback device capable of supporting the ASTC format in order to solve the heat problem with the ASTC image format with less CPU resource use. On the other hand, in the case of a problem in which an HMD status of an experienced user cannot be checked and controlled, it is related to an error check whether the content is normally operated and a lack of a control system. Accordingly, in order to increase operational convenience and stability, a plurality of mobile HMDs (that is, VR playback devices) are monitored in real time and controllable management applications are developed.

Briefly, the integrated management system includes an ASTC VR playback device, a monitoring/control device and a motion simulator. The ASTC VR playback device may include an ASTC sequence file, an ASTC sequence file and synchronized audio files and motion data for cinematic VR play. Current playback information of the ASTC VR playback device can include real-time frame control information, real-time X, Y, and Z-axis tilt values, external sensor data, basic output data, real-time frame count values, battery information, and fever temperature check information. have. In addition, the ASTC VR playback device is equipped with a communication interface, e.g., a Bluetooth interface, to receive advertising signals from a monitoring/control device, exchange master/slave data, and/or a WIFI interface with a WIFI interface. By exchanging data through, motion sink data can be output to the motion simulator. The monitoring/controlling device can monitor a real-time video of an experienced person wearing a VR playback device and link the video with the player in real time. The monitoring/control device can individually or collectively control playback of a plurality of ASTC VR playback devices. In this regard, the monitoring/control device is equipped with a communication interface, for example, a Bluetooth interface, to transmit advertising signals to collectively control a plurality of ASTC VR playback devices within a broadcasting range, or to control master/slave data. You can individually control the ASTC VR playback device through a 1:1 connection by exchanging and/or individually control the ASTC VR playback device through a 1:1 connection by exchanging data using WIFI with a WIFI interface. Can. The monitoring/control device can also output motion sink data to the motion simulator. The motion simulator is equipped with a communication interface, such as a Bluetooth interface or a WIFI interface, to receive motion sync data from an ASTC VR playback device and a monitoring/control device.

1 is a schematic diagram of an integrated management system 100, according to at least some embodiments of the present disclosure. The integrated management system 100 may include one or more playback devices 100 for virtual reality content and an integrated management device 120 that is communicatively connected to the playback devices 100 and monitors each of them. The integrated management system 100 may further include a motion simulator 130 communicatively connected to the playback apparatus 100 and the integrated management device 120.

The playback device 110 is a device of various types to be developed currently or in the future, mainly using the HMD type, and is mounted on the user's head to provide a stimulus capable of visual and/or auditory as well as experience of other virtual reality content. It may include a device. Basically, currently widely used devices can provide video and audio data to users, which are downloaded as a virtual reality content from a virtual reality content provider and provided to users through HMD's display and speakers. Is done. The HMD may also be an interactive device capable of controlling virtual reality content by receiving user input such as a user's voice or a user's gaze direction.

In addition, in order to reduce the user's convenience and the installation cost of the operator, the playback device 110 may be used by mounting a user terminal on an HMD type VR device. The user terminal is, for example, a mobile phone, a smart phone, a personal data assistant (PDA), a wearable device such as a personal media player device, a wireless web-watch device, a personal headset device, a special purpose device, or any of the above functions It may be implemented as a portable (or mobile) electronic device of a small form factor (small-form factor), such as a hybrid device comprising a.

The integrated management device 120 is a device for controlling and monitoring one or more playback devices 110. For this, at least a device having a basic hardware configuration such as a memory or a processor is sufficient. The integrated management device 120 may include an integrated management server and a display, and the integrated management server may be any computing device having an application server, a standalone server, a web server, and other data transmission/reception functions, data identification functions, and data processing functions. Can.

The motion simulator 130 may provide, for example, a linear motion, a vertical motion, a rotational motion, a swing motion, and the like to a user who is on board with an actuator. The motion simulator 130 may have various degrees of freedom. Depending on the type of virtual reality content provided, the motion simulator 130 may further include, for example, a wind generator and a water jet device.

Next, the interaction on the integrated management system 100 between the playback apparatus 110, the integrated management device 120, and the motion simulator 130 will be described. The playback device 110 may analyze the virtual reality content and extract image data, audio data, and/or motion data. 2 is a diagram illustrating an example of a configuration of virtual reality content capable of cinematic VR play. FIG. 2(a) shows image data composed in units of frames including an ASTC sequence file. 2(b) shows an audio file in the form of a waveform synchronized with image data. 2(c) shows motion data having motion values synchronized with image data.

The playback device 110 detects the format of the video data, determines whether it can be supported, and when the playback device 110 determines that it can be supported, processes the video data based on the detected format and outputs it as a frame-by-frame video. . The playback device 110 may support video data in various formats, such as MP4 format and ASTC format. The reproducing apparatus 110 may reproduce the image-processed image data in units of frames, count the number of regenerated frames, calculate the current frame number of the image data, and use it as reproducing information of the reproducing apparatus 110. The playback device 110 may calculate and use at least one of fps, heat generation temperature, CPU usage, memory usage, and battery level during playback, as status information of the playback apparatus 110.

The integrated management device 120 may receive the current frame number of the video data being played from each playback device 110 and monitor the playback state of each playback device 110. The integrated management device 120 may monitor at least one of fps, heat generation temperature, CPU usage, memory usage, and remaining battery power from each playback apparatus 110 to monitor the usage status of each device. The integrated management device 120 transmits an advertising signal to the playback device 110, exchanges master/slave data, or exchanges WIFI data, for example, by Bluetooth, to play one or more playback devices 110. You can control Through the control of the integrated management device 120, the playback frames of the plurality of playback devices 110 may be synchronized.

The motion simulator 130 may be communicatively connected to the playback apparatus 110 and the integrated management device 120 by various connection methods, including wireless connection and wired connection, respectively. The motion simulator 130 receives motion data having a synchro motion value equal to a frame unit of image data from the playback apparatus 110 or the integrated management device 120, and performs a specific movement when playing each frame according to the motion value. Can provide.

3 is a diagram illustrating an example of a playback device 300 for virtual reality content, according to at least some embodiments of the present disclosure. The playback device 300 may include a content search unit 310 and a content selection unit 315. The content search unit 310 may search for virtual reality content stored in the playback device. The content selection unit 315 may display a playable virtual reality content according to a search result of the content search unit and provide a user interface for selecting virtual reality content to be played. When the user selects and inputs the virtual reality content, the playback device 300 may be played based on the input value.

The playback device 300 may include a content analysis unit 320 that analyzes virtual reality content stored in the playback device 300. The content analysis unit 320 may analyze virtual reality content and extract image data, audio data, and motion data. Here, the audio data may have the same sync audio section as the frame unit of the video data, and the motion data may have the same sync motion value as the frame unit of the video data. Image data and audio data may be used as basic output data of the playback device 300.

The playback device 300 may include a format detection unit 325 that detects the format of the video data extracted by the content analysis unit 320 and determines whether the playback device 300 can support it. For example, the playback device 300 may support both the adaptive scalable texture compression (ASTC) sequence file format and the MP4 file format. The format detection unit 325 may generate a support error message when the format of the video data cannot be supported by the playback device 300.

The playback apparatus 300 may include an image processing unit 330 for processing image data based on the detected format and outputting it as a frame-by-frame image when it is determined that the playback apparatus can support it. The playback device 300 may include an audio processing unit 330 for processing and outputting audio data having the same sync as the video data. The playback device 300 may include a display unit 340 for reproducing image processed image data in units of frames. For example, video data in the ASTC sequence file format is reproduced in a 30 fps sequence, and the maximum loading resolution may be 8K. The playback device 300 may include a speaker (not shown) that generates sound waves based on audio data.

The playback device 300 may include a frame management unit 345 that counts the number of played frames to calculate the current frame number of image data. The regeneration device 300 may include a temperature sensing unit 350 that senses the heat generation temperature of the regeneration device 300. For example, the heating temperature of the regeneration device 300 may be a CPU temperature or a battery temperature built into the regeneration device 300, which may be used as the internal temperature of the regeneration device 300.

The playback device 300 may include a usage checking unit 355. The usage checking unit 355 may check at least one of the CPU usage, memory usage, and battery level of the playback device. The CPU usage, memory usage, and battery residual amount may be used as status information of the playback device 300.

The playback device 300 may include an eye tracking unit 360. The target tracking unit 360 may recognize the user's gaze of the playback device and calculate an input value. The target tracking unit 360 may measure the position and movement of the user's eye equipped with the playback device 300. For example, when the user looks at a button to be selected from the user interface, the target tracking unit 360 may recognize a button facing the user's gaze as an input. Also, the reproduction device 300 may include a tilt detection unit (not shown). The tilt detection unit may be, for example, a gyro sensor. The tilt detection unit may detect the real-time X, Y, and Z-axis tilt of the HMD-type playback device 300 mounted on the user's head and calculate an input value. The real-time X, Y, and Z-axis tilt can be reflected in the result of the screen to be displayed on the user's HMD.

The playback device 300 may include a communication unit 365 communicatively connected to an integrated management device that controls and monitors the motion simulator and the playback device 300. The communication unit 365 may transmit motion data identical to the VR image to the motion simulator. The communication unit 365 may transmit playback information or status information of the playback device 300 to the integrated management device in real time. For example, the communication unit 365 may transmit at least one of a current frame number, fps (frame per second), fever temperature, CPU usage, memory usage, and battery remaining amount when the playback apparatus is played back to the integrated management device. The communication unit 365 may receive external sensor data. The external sensor data is a value recognized by various external sensors, such as data about a user's step or joystick data in a virtual reality device of the type treadmill, and can be used as an input value for virtual reality content. Various types of wired or wireless connection methods may be used as the communication unit 365, but wireless communication may be basically used for the convenience of an experienced user. For example, the communication unit 365 may receive an advertising signal from the integrated management device as a Bluetooth interface, exchange master/slave data, or exchange data via WIFI with the integrated management device as a WIFI interface.

The playback device 300 according to the present invention is characterized in that it is capable of supporting a sequence file in the ASTC format. When using the ASTC VR playback device as in the present invention, it may be provided with improved quality compared to the GPU format, and the maximum loading resolution may be increased to 8K. In addition, it is possible to obtain an effect of reducing the heat generation of the device by using OpenGL. In addition, compared to a conventional MP4 format playback device, using about 70% of memory has an advantage in terms of memory usage, and can also increase the battery usage time. The ASTC VR playback device according to the present invention can be played back in a 30 fps sequence.

Hereinafter, development functions of a playback device that can be implemented by an actual developer are presented as Table 1 and Table 2, but this is a problem according to the developer's selection, and the function of the playback device is not limited to these functions. Here, Table 1 shows the development function of the playback device when the playback device targets B2C, and Table 2 shows the development function of the playback device when the playback device targets B2B.

4 is a diagram illustrating an example of a monitoring screen 400 of an integrated management device, according to at least some embodiments of the present disclosure. The integrated management device is implemented to enable real-time monitoring and real-time interlocking of the content played in the playback device of the experienced person, including a status monitoring and control application of the playback device. The integrated management device can monitor and control multiple playback devices respectively.

The monitoring screen 400 may include a portion 410 displaying the number of playback devices currently being played. For example, the number of playback devices currently being played on the monitoring screen 400 is four. In order to monitor and control the second playback device, the manager selected the part marked with 2 to distinguish colors differently from the unselected part (ie, parts marked with 1, 3, and 4). The monitoring screen 400 may display the status information of the VR playback device so that an administrator can check it. For example, the monitoring screen 400 is status information of the VR playback device, and may include a current video frame count part 420, a battery remaining part 430, and an internal temperature part 440 of the VR playback device. Here, the display of the current image frame count portion 420 is 3275, the battery remaining portion 430 is 92%, and the internal temperature portion 440 is 35 degrees. Also, the monitoring screen 400 may display an image portion 425 corresponding to the current frame. Here, an image corresponding to frame number 3275 is displayed on the screen. Image data provided for the display of the image portion 425 is not received by each VR playback device. The integrated management device may have frame information and image information corresponding to the frame information, and receive only the current frame number from the VR playback apparatus to provide the image corresponding thereto. The monitoring screen 400 may include a reproduction management part 450. The playback management part 450 is included so as to individually or collectively control the VR playback devices of a plurality of experienced users. For example, the playback management part 450 may be made to allow an administrator to select functions such as playback, pause, and pause of an image. The main performance values are: 1) Real-time interlocking time between the playback device and the monitoring/control application video is within 0.1 seconds, 2) Replay signal transmission speed / current frame interlocking speed is within 0.1 seconds, 3) Battery level / interlocking speed of HMD internal temperature display Can be implemented to be within 0.1 second, so that multiple VR playback devices can be controlled and monitored quickly and without error.

Hereinafter, the development function of the integrated management device that can be implemented by the actual developer is presented as Table 3, but this is a problem according to the developer's choice, and the function of the integrated management device is not limited to these functions.

5 is a flow diagram 500 illustrating an exemplary process of a method for monitoring one or more playback devices for virtual reality content under the control of an integrated management server, according to at least some embodiments of the present disclosure.

Processing can begin at block 502 that sends a playback signal to the playback device. The integrated management server or the integrated management device may transmit a playback signal to the playback apparatus for causing the playback apparatus to play virtual reality content in units of frames. Here, the integrated management server or the integrated management device transmits an advertising signal to the playback device, for example, via a Bluetooth interface, or exchanges master/slave data, and/or exchanges WIFI data through a WIFI interface. You can control the playback of one or more playback devices.

Processing may continue at block 502 to block 504 that sends motion data to the motion simulator. The integrated management server or the integrated management device may transmit the motion data to the motion simulator while simultaneously transmitting the playback signal to the playback device. Here, the motion data may be a sync motion value equal to a frame unit of image data.

Processing may continue at block 504 to a block 506 that receives the current frame number from the playback device. The integrated management server or the integrated management device may respectively receive current frame numbers of video data being played from one or more playback devices. For example, the current frame number may include, for example, a sequence number of the ASTC sequence file. That is, if the ASTC format is supported by the playback device, the current frame number being played may be the current sequence number of the ASTC sequence file being played. In one embodiment, the integrated management server or the integrated management device may calculate the maximum frame number among the current frame numbers of the plurality of playback devices, and synchronize the plurality of playback devices based on the calculated maximum frame number. That is, in order to synchronize frames of the entire playback device connected to the integrated management server or integrated management device, the integrated management server or integrated management device calculates the current frame number according to the earliest frame, and based on this, each playback device You can order. This synchronization function of the entire playback device may be particularly useful when an experienced user of a plurality of playback devices needs to experience virtual reality content at the same time.

Processing may continue at block 506 to block 508 displaying an image corresponding to the current frame number. Here, the integrated management server and the integrated management device can basically provide a low-quality thumbnail of the current playback image corresponding to the current frame number of the entire registered playback device, and when the administrator selects a specific playback device on the monitoring screen, the current The current playback image corresponding to the frame number can be provided in full screen with high quality.

Processing may continue at block 508 to a block 510 that receives usage status information from the playback device. The integrated management server and the integrated management device may receive at least one of fps, heat generation temperature, CPU usage, memory usage, and battery level during reproduction as usage state information of the reproduction apparatus.

Processing may continue at block 510 to block 512 to determine if usage status information is within a range of predetermined values for each. For example, the usage status information may be at least one of fps, heat generation temperature, CPU usage, memory usage, and battery remaining during reproduction. Each usage status information may be set in an allowable range for each of them.

Processing may continue to block 514 to generate a warning message if it is determined at block 512 that one or more of the usage status information is not within a range of predetermined values. For example, the integrated management server and the integrated management device may have fps lower than a certain minimum fps during playback, heat generation temperature higher than a certain maximum temperature, CPU usage greater than a certain maximum allowable CPU usage, or memory usage a specific maximum allowable memory. If it is greater than the usage amount and/or the battery level is lower than a certain minimum battery level, it may be determined that it is not within a predetermined range of values and generate a warning message.

The process 500 by the integrated management server shown in FIG. 5 does not limit the technical spirit of the present disclosure to such illustrative examples due to the description of the process blocks. For example, those skilled in the art will understand that a modified process is possible by variously changing or combining the order and function of each block of the present process disclosed herein. On the other hand, the schematic operations illustrated in FIG. 5 are provided as examples only, and without departing from the essence of the embodiment, selectively remove some of the operations, combine them with fewer operations, or expand to include additional operations I can do it.

Among other possible advantages, a VR playback device capable of supporting ASTM format according to at least some embodiments of the present disclosure may be utilized as a mobile HMD universal VR video player that provides high-definition and high-resolution VR video. In addition, while reducing the heat generation phenomenon of the mobile HMD, which has been a conventional problem, and extending the battery usage time, there is an effect that a stable service can be provided by reducing the memory storage capacity in the mobile device.

In addition, the integrated management system equipped with the HMD status monitoring and control application according to at least some embodiments of the present disclosure is expected to be used as an urban VR theme park operating system, which can greatly increase the operational convenience of the VR theme park. As the time required for inspecting the experienced equipment required for the conventional administrator to check the normal operation state of a large number of VR devices decreases, the experience rotation rate may increase. In addition, since the error incidence rate is reduced, operational stability can be secured, which can lead to increased sales.

The above description of the present application is for illustrative purposes, and a person having ordinary knowledge in the technical field to which the present application belongs will understand that it is possible to easily change to other specific forms without changing the technical spirit or essential characteristics of the present application. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

In the above, the object to be claimed in the present disclosure has been described in detail. The subject matter claimed in this disclosure is not to be limited in scope by the specific embodiments described above. For example, in some implementations it may be in the form of hardware operably used on a device or combination of devices, in other implementations may be implemented in the form of software and/or firmware, in other implementations a signal bearing medium, It may include one or more articles, such as storage media. Here, a storage medium such as a CD-ROM, computer disk, flash memory, or the like, when executed by a computing device, such as a computing system, computing platform, or other system, may cause instructions to be executed according to the above-described implementation. Can be saved. Such computing devices may include one or more processing units or one or more input/output devices such as processors, displays, keyboards and/or mice, and one or more memories such as static random access memory, dynamic random access memory, flash memory and/or hard drives. It may include.

On the other hand, whether the system is implemented in hardware or software is generally a design selection problem that represents a cost-effective tradeoff. In the present disclosure, there are various means (e.g., hardware, software and/or firmware) that can be affected by processes, systems, and other technologies, and preferred means are used by processes and/or systems and/or other technologies. It will change depending on the context in which it is made. For example, if the implementer determines that speed and accuracy are the most important, the implementer can primarily choose the hardware and/or firmware means, and if flexibility is the most important, the implementer can mainly choose the software implementation; Or, as another alternative, implementers may choose any combination of hardware, software and/or firmware.

In the foregoing detailed description, various embodiments of apparatus and/or processes have been described through block diagrams, flowcharts, and/or other examples. Such block diagrams, flow diagrams, and/or other examples will include one or more functions and/or operations, and those skilled in the art will appreciate that each function and/or operation in the block diagrams, flow charts, and/or other examples is hardware, software, firmware, Or it will be understood that it can be implemented individually or collectively by any combination of these. In one embodiment, some portions of the subject matter described in this disclosure may be implemented through an application specific integrated circuit (ASIC), field programmable gate array (FPGA), digital signal processor (DSP), or other form of integration. Alternatively, some aspects of the embodiments of the present disclosure include one or more computer programs (eg, one or more programs running on one or more computer systems) running on one or more computers, one or more programs running on one or more processors ( For example, one or more programs running on one or more microprocessors), firmware, or substantially any combination thereof, may be implemented in integrated circuits, in whole or in part, equally, or written in code for software and/or firmware And/or the design of the circuit is within the skill of the artisan in light of the present disclosure. Further, those skilled in the art will understand that the mechanisms of the subject matter of the present disclosure can be distributed into various types of program products, and the examples of the subject matter of the present disclosure apply regardless of the specific type of signal bearing medium used to actually perform the distribution. Will understand

Although certain exemplary techniques have been described and illustrated herein using a variety of methods and systems, those skilled in the art can understand the possibility of substitution with various other modifications or equivalents without departing from the claimed subject matter. Additionally, many modifications can be made to adapt a particular situation to the teaching of the claimed subject without departing from the central concept described herein. Accordingly, it is intended that the claimed subject matter is not limited to the particular examples disclosed, but such claimed subject matter may also include all embodiments falling within the scope of the appended claims and their equivalents.

Throughout this disclosure, when a part is said to be "connected" to another part, this includes not only "directly connected" but also "electrically connected" with other elements in between. do. Further, throughout this disclosure, when one member is said to be "on" another member, this includes not only the case where one member is in contact with the other member but also another member between the two members. Furthermore, throughout the present disclosure, when a part “includes” a certain component, it means that the component may further include other components, not to exclude other components, unless specifically stated to the contrary. The terms “about,” “substantially,” and the like, as used in the present disclosure, are used in or near the numerical values when manufacturing and material tolerances unique to the stated meanings are presented, to aid understanding of the present application. Hazards are used to prevent unreasonable abuse by unscrupulous infringers of the disclosures that are either accurate or absolute.

The scope of the present disclosure is indicated by the following claims rather than the above detailed description, and it should be interpreted that all changes or modified forms derived from the meaning and scope of the claims and equivalent concepts thereof are included in the scope of the present application. do.

Claims (20)

As a playback device for virtual reality content,
A content analysis unit that analyzes the virtual reality content stored in the playback device and extracts image data;
A format detection unit that detects the format of the image data and determines whether it can be supported by the playback device;
An image processing unit for processing the image data based on the detected format and outputting it as a frame-by-frame image when it is determined that the playback apparatus can support the image;
A display unit for reproducing the image-processed image data in units of frames;
A frame management unit counting the number of reproduced frames to calculate a current frame number of the image data; And
Temperature sensing unit for sensing the heating temperature of the playback device based on the battery temperature of the playback device
Including,
The playback device is capable of supporting image data in an adaptive scalable texture compression (ASTC) format. delete delete According to claim 1,
The playback device further includes a usage check unit,
The usage confirmation unit is to check at least one of the CPU usage, memory usage and battery level of the reproduction device. According to claim 1,
The playback device further includes a content search unit and a content selection unit,
The content search unit searches the virtual reality content stored in the playback device,
The content selection unit displays a playable virtual reality content according to a search result of the content search unit, and provides a user interface for selecting virtual reality content to be played. According to claim 1,
The reproducing device further includes an eye tracking unit,
The target tracking unit calculates an input value by recognizing a user's gaze of the playback device. According to claim 1,
The playback device further includes an audio processing unit for processing and outputting audio data,
The content analysis unit extracts the audio data by analyzing the virtual reality content,
The audio data is the same as the frame unit of the video data, the sync audio section is determined, the playback device. According to claim 1,
The playback device further includes a communication unit communicatively connected to the motion simulator,
The content analysis unit extracts motion data by analyzing the virtual reality content,
The communication unit transmits the motion data to the motion simulator,
The motion data has a sync motion value equal to a frame unit of the video data, and a reproduction device. According to claim 1,
The playback device further includes a communication unit communicatively connected to an integrated management device for controlling and monitoring the playback device,
The communication unit transmits at least one of a current frame number, fps (frame per second), fever temperature, CPU usage, memory usage, and battery level when the reproduction apparatus is played back to the integrated management device. As an integrated management system,
One or more playback devices for virtual reality content; And
An integrated management device that is communicatively connected to the playback device and monitors each of them
Including,
The playback device,
Supports ASTC format video data,
Analyzing the virtual reality content to extract image data,
Detecting the format of the video data, determining whether the playback device can support it,
When it is determined that the playback device can support, the image data is processed based on the detected format and output as a frame-by-frame image,
The video data, which has been processed, is reproduced in units of frames,
Count the number of reproduced frames to calculate the current frame number of the video data, and
Sensing the heating temperature of the regeneration device based on the battery temperature of the regeneration device,
The integrated management device is to monitor the playback status of each playback device by receiving the current frame number of the video data being played from each playback device, the integrated management system. delete The method of claim 10,
The reproducing device calculates at least one of fps, heat generation temperature, CPU usage, memory usage, and battery remaining during reproduction,
The integrated management device is to monitor the usage status of each device by receiving at least one of the fps, heat generation temperature, CPU usage, memory usage and battery level from each playback device. The method of claim 10,
The integrated management device controls the playback of the one or more playback devices by transmitting an advertising signal to the playback device, exchanging master/slave data, or exchanging WIFI data. The integrated management system. The method of claim 10,
The integrated management system further includes a motion simulator each communicatively connected to the playback device and the integrated management device,
The motion simulator receives motion data having a synchro motion value equal to a frame unit of the video data from the playback device or the integrated management device, and provides a specific motion when playing each frame according to the motion value. , Integrated management system. A method for monitoring one or more playback devices for virtual reality content capable of supporting ASTC format image data under the control of an integrated management server,
Transmitting, to the playback device, a playback signal for causing the playback device to play the virtual reality content on a frame-by-frame basis;
Receiving a current frame number of video data being played back from the playback device;
Receiving a heat generation temperature of the regeneration device sensed based on a battery temperature of the regeneration device; And
Displaying an image corresponding to the current frame number
Method for monitoring comprising a. The method of claim 15,
The method for monitoring,
The method for monitoring further comprising the step of transmitting the motion data having the same sync motion value as the frame unit of the image data to the motion simulator simultaneously with the transmission of the reproduction signal. The method of claim 15,
The method for monitoring,
Receiving, from the playback device, at least one of fps, heat generation temperature, CPU usage, memory usage, and remaining battery capacity during reproduction as usage state information of the reproduction apparatus;
Determining whether the usage status information is within a range of a predetermined value for each; And
And if it is determined that at least one of the usage status information is not within the range of the predetermined value, generating a warning message. The method of claim 15,
The step of transmitting the playback signal includes controlling an playback of the one or more playback devices by transmitting an advertising signal to the playback device, exchanging master/slave data, or exchanging WIFI data. Way to do. The method of claim 15,
The step of receiving the current frame number includes receiving a sequence number of an ASTC sequence file. The method of claim 15,
The step of receiving the current frame number,
Calculating a maximum frame number among current frame numbers of a plurality of playback devices; And
Synchronizing the plurality of playback devices based on the maximum frame number
A method for monitoring, comprising:

Images