WO2020158095A1 - Dispositif d'évaluation - Google Patents

Dispositif d'évaluation Download PDF

Info

Publication number
WO2020158095A1
WO2020158095A1 PCT/JP2019/043656 JP2019043656W WO2020158095A1 WO 2020158095 A1 WO2020158095 A1 WO 2020158095A1 JP 2019043656 W JP2019043656 W JP 2019043656W WO 2020158095 A1 WO2020158095 A1 WO 2020158095A1
Authority
WO
WIPO (PCT)
Prior art keywords
evaluation
reproduction
unit
moving image
processing device
Prior art date
Application number
PCT/JP2019/043656
Other languages
English (en)
Japanese (ja)
Inventor
一輝 浅井
潤相 呉
銀平 岡田
Original Assignee
株式会社Ｎｔｔドコモ
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社Ｎｔｔドコモ filed Critical 株式会社Ｎｔｔドコモ
Publication of WO2020158095A1 publication Critical patent/WO2020158095A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/258—Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording
    • H04N5/91—Television signal processing therefor
    • H04N5/93—Regeneration of the television signal or of selected parts thereof

Definitions

  • the present invention relates to an evaluation device.
  • Patent Document 1 describes a user terminal that reproduces a moving image based on moving image data transmitted by a distribution site.
  • a user terminal as described in Patent Document 1 reproduces a moving image by executing software for reproducing the moving image based on the moving image data (hereinafter referred to as “moving image reproducing software”).
  • the video playback software is provided to the user terminal after passing the test in the development stage.
  • An object of the present invention is to provide a technology that can reduce the time and effort required for testing the moving image playback software during the development stage.
  • An evaluation device uses a learning model that has learned relational data indicating a relationship between a situation regarding reproduction of a moving image and an evaluation for the situation, and performs reproduction regarding reproduction of a moving image performed by software for reproducing a moving image.
  • An evaluation unit that evaluates the situation and an output unit that outputs information based on the evaluation result of the evaluation unit are included.
  • FIG. 1 is a diagram showing an overall configuration of an evaluation system 1 according to the first embodiment.
  • the evaluation system 1 evaluates the reproduction software at the development stage of the reproduction software for reproducing the moving image and the sound.
  • the playback software is an example of moving image playback software.
  • the evaluation system 1 includes a distribution server 10, an information processing device 20, and an evaluation device 30.
  • the distribution server 10, the information processing device 20, and the evaluation device 30 can communicate with each other via the network NW.
  • the number of each of the distribution server 10, the information processing device 20, and the evaluation device 30 is not limited to “1”.
  • the evaluation system 1 may include a plurality of distribution servers 10, a plurality of information processing devices 20, and a plurality of evaluation devices 30.
  • the distribution server 10 transmits distribution data including moving image data indicating a moving image and sound data indicating sound to the user device 20.
  • the distribution data is, for example, data representing a movie.
  • the distribution data is not limited to data representing a movie, but may be data representing a television program, for example.
  • Each of the moving image data and the audio data is encoded data.
  • the reproduction timing of the moving image based on the moving image data and the reproduction timing of the sound based on the audio data are defined in advance.
  • the frame rate of the moving image indicated by the moving image data is fixed to the reference frame rate.
  • the reference frame rate is, for example, 30 fps (Frames Per Second).
  • the reference frame rate is not limited to 30 fps and may be higher or lower than 30 fps.
  • the frame rate of the moving image indicated by the moving image data may be changed.
  • the distribution server 10 adjusts the bit rate of distribution data in response to a request from the user device 20.
  • the distribution server 10 is composed of a computer system including a communication device 101, a storage device 102, and a processing device 103.
  • the term “device” in a communication device, a storage device, a processing device, an input device described below, and a display device described below may be replaced with another term such as a circuit, a device, or a unit.
  • the respective elements of the distribution server 10 are mutually connected by a single bus or a plurality of buses. Each element of the distribution server 10 is composed of one or more devices.
  • the communication device 101 communicates with another device, for example, the information processing device 20, via the network NW.
  • the communication device 101 is also called, for example, a network device, a network controller, a network card, or a communication module.
  • the storage device 102 is a recording medium that can be read by the processing device 103.
  • the storage device 102 stores a plurality of programs including a control program executed by the processing device 103, various data used by the processing device 103, and various distribution data.
  • the storage device 102 is configured by at least one of a recording medium such as a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable ROM), and a RAM (Random Access Memory).
  • the processing device 103 is a processor that controls the distribution server 10.
  • the processing device 103 is composed of, for example, one or more chips.
  • the processing device 103 includes an interface with peripheral devices and a central processing unit (CPU).
  • the central processing unit includes an arithmetic unit, a register and the like. Even if some or all of the functions of the processing device 103 are executed by hardware such as DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), FPGA (Field Programmable Gate Array), etc. Good.
  • the processing device 103 executes various processes in parallel or sequentially.
  • the processing device 103 reads the control program from the storage device 102.
  • the processing device 103 functions as the distribution control unit 110 by executing the control program.
  • the delivery control unit 110 is an example of a functional block of the processing device 103.
  • the distribution control unit 110 may be configured by hardware such as DSP, ASIC, PLD, or FPGA.
  • the delivery control unit 110 controls transmission of delivery data.
  • the distribution control unit 110 transmits distribution data to the information processing device 20 in response to a request from the information processing device 20.
  • the distribution control unit 110 also adjusts the bit rate of the moving image data included in the distribution data in response to a request from the information processing device 20.
  • the bit rate of moving image data represents the amount of moving image data transmitted per second.
  • One second is an example of unit time.
  • the unit time is not limited to 1 second and may be longer or shorter than 1 second.
  • the information processing device 20 is, for example, a smartphone.
  • the information processing device 20 is not limited to the smartphone.
  • the information processing device 20 may be a notebook computer, a wearable terminal, a tablet terminal, a desktop computer, or the like.
  • the information processing device 20 receives the distribution data from the distribution server 10. Playback software at the development stage is installed in the information processing device 20.
  • the information processing device 20 reproduces the moving image and the sound by processing the distribution data using the reproduction software in the development stage.
  • the information processing device 20 is configured by a computer system including an input device 201, a display device 202, a speaker 203, a communication device 204, a storage device 205, and a processing device 206. Each element of the information processing device 20 is mutually connected by one or more buses. Further, each element of the information processing device 20 is configured by one or more devices.
  • the input device 201 is a device for inputting information used by the information processing device 20 to the information processing device 20.
  • the input device 201 receives user input such as instructions or questions from the user.
  • the input device 201 receives an operation for inputting codes such as numbers and letters to the information processing device 20 and an operation for selecting an icon displayed by the display device 202.
  • the input device 201 is, for example, a touch panel that detects contact with the display surface of the display device 202.
  • the input device 201 is not limited to the touch panel.
  • the input device 201 may be a keyboard, a mouse, a switch, a button, or the like.
  • the display device 202 displays various images under the control of the processing device 206.
  • the display device 202 displays the icon and the moving image based on the moving image data included in the distribution data at the same time or at different timings.
  • the display device 202 is, for example, a liquid crystal display panel.
  • the display device 202 is not limited to the liquid crystal display panel.
  • the display device 202 may be an organic EL (Electro Luminescence) display panel or the like.
  • the speaker 203 outputs various sounds under the control of the processing device 206. For example, the speaker 203 outputs a sound based on the sound data included in the distribution data.
  • the communication device 204 communicates with other devices, for example, the distribution server 10 and the evaluation device 30 via the network NW.
  • the communication device 204 is also called, for example, a network device, a network controller, a network card, or a communication module.
  • the storage device 205 is a recording medium that can be read by the processing device 206.
  • the storage device 205 stores a plurality of programs executed by the processing device 206 and various data used by the processing device 206.
  • the plurality of programs executed by the processing device 206 include reproduction software at the development stage.
  • the storage device 205 includes, for example, a non-volatile memory and a volatile memory.
  • the non-volatile memory is, for example, ROM, EPROM, or EEPROM.
  • the volatile memory is, for example, a RAM.
  • the volatile memory included in the storage device 205 is used as a working memory of the processing device 206.
  • the processing device 206 is a processor that controls the information processing device 20.
  • the processing device 206 includes, for example, one or more chips.
  • the processing unit 206 is configured by a central processing unit (CPU).
  • CPU central processing unit
  • Some or all of the functions of the processing device 206 may be realized by hardware such as DSP, ASIC, PLD, or FPGA.
  • the processing device 206 executes various processes in parallel or sequentially.
  • the processing device 206 reads a plurality of programs from the storage device 205.
  • the processing device 206 functions as the reproduction executing unit 210 and the operation control unit 220 by executing the plurality of programs.
  • the processing device 206 reads the reproduction software at the development stage from the storage device 205.
  • the processing device 206 functions as the reproduction executing unit 210 by executing the reproduction software at the development stage.
  • the reproduction execution unit 210 and the operation control unit 220 are examples of functional blocks of the processing device 206.
  • the operation control unit 220 may be configured by hardware such as DSP, ASIC, PLD or FPGA.
  • the reproduction execution unit 210 is realized based on reproduction software at the development stage.
  • the reproduction execution unit 210 reproduces a moving image and a sound based on the distribution data.
  • the reproduction executing unit 210 decodes distribution data received from the distribution server 10 by the communication device 204, specifically, each of moving image data and audio data.
  • the reproduction executing unit 210 provides the display device 202 with the image information that is the decoding result of the moving image data, thereby causing the display device 202 to display the moving image indicated by the moving image data.
  • the reproduction executing unit 210 provides the speaker 203 with the audio information that is the decoding result of the audio data, so that the speaker 203 outputs the audio indicated by the audio data.
  • the reproduction executing unit 210 first decrypts the distribution data. Subsequently, the reproduction executing unit 210 generates image information and audio information by decoding the decrypted distribution data.
  • DRM Digital Rights Management
  • the operation control unit 220 controls the operation of the information processing device 20. For example, the operation control unit 220 transmits, from the communication device 204 to the evaluation device 30, a reproduction log indicating a situation regarding reproduction of a moving image in the information processing device 20.
  • the reproduction log is an example of status information.
  • the playback log indicates the status of the number of unplayed image frames (hereinafter referred to as “non-playback image frames”) among the consecutive image frames that make up the video based on the video data, as the status related to the video playback.
  • the reproduction log indicates the number of non-reproduction image frames, for example, the number of continuous non-reproduction image frames (hereinafter referred to as “number of consecutive frame drops”).
  • the number of consecutive frames dropped can also be referred to as “the number of consecutive unreproduced image frames”. As an example, if the image frames with index numbers “3” to “8” are not played back among the nine consecutive image frames with index numbers “1” to “9”, the playback log is "6" is shown as the number of drops.
  • the reproduction log indicates a situation regarding the reproduction of the moving image
  • sound and moving image a time difference between the reproduction timing of the sound to be reproduced together with the moving image (the sound reproduced based on the sound data) and the moving image (hereinafter referred to as “sound and moving image”).
  • sound and moving image a time difference between the reproduction timings of the audio and the moving image means the discrepancy time between the reproduction timing of the audio and the actual reproduction timing of the moving image to be reproduced in synchronization with the reproduction timing of the audio.
  • the time difference between the reproduction timings of the sound and the moving image is an example of the situation of the difference between the reproduction timings of the sound and the moving image.
  • the status of the number of unplayed image frames, the number of consecutive frames dropped, the status of audio/video playback timing deviation, and the audio/video playback timing deviation time are each related to video playback disturbance. Is an example.
  • the reproduction log of the information processing device 20 indicates the number of consecutive frames dropped and the time difference between the reproduction timings of the audio and the moving image.
  • the reproduction log of the information processing device 20 will be referred to as “evaluation reproduction log”.
  • Evaluation device 30 The evaluation device 30 automatically evaluates the reproduction software at the development stage based on the reproduction log for evaluation (reproduction log of the information processing device 20).
  • the evaluation device 30 is composed of a computer system including a communication device 301, a display device 302, a speaker 303, a storage device 304, and a processing device 305.
  • the elements of the evaluation device 30 are connected to each other by one or more buses.
  • each element of the evaluation device 30 is configured by one or two or more devices.
  • the communication device 301 communicates with another device, for example, the information processing device 20, via the network NW.
  • the communication device 301 is also called, for example, a network device, a network controller, a network card, or a communication module.
  • the display device 302 displays various images under the control of the processing device 305.
  • the display device 302 displays an image showing the evaluation result of the reproduction software at the development stage.
  • the display device 302 is, for example, a liquid crystal display panel.
  • the display device 302 is not limited to a liquid crystal display panel.
  • the display device 302 may be an organic EL display panel or the like.
  • the speaker 303 outputs various sounds under the control of the processing device 305.
  • the speaker 303 informs the evaluation result of the reproduction software at the development stage by sound.
  • the speaker 303 may be omitted.
  • the storage device 304 is a recording medium that can be read by the processing device 305.
  • the storage device 304 stores a plurality of programs including a control program executed by the processing device 305, and various data used by the processing device 305.
  • the storage device 304 also stores a learning model 304a including a plurality of coefficients K.
  • the storage device 304 is configured by at least one of recording media such as ROM, EPROM, EEPROM, and RAM.
  • the processing device 305 is a processor that controls the evaluation device 30.
  • the processing device 305 includes, for example, one or more chips.
  • the processing device 305 is configured by a central processing unit (CPU).
  • CPU central processing unit
  • Some or all of the functions of the processing device 305 may be realized by hardware such as DSP, ASIC, PLD, or FPGA.
  • the processing device 305 executes various processes in parallel or sequentially.
  • the processing device 305 reads the learning model 304a and the control program from the storage device 304, for example.
  • the processing device 305 functions as the receiving unit 310, the evaluation unit 320, and the output unit 330 by executing the learning model 304a and the control program.
  • the evaluation unit 320 is realized by executing the learning model 304a.
  • the receiving unit 310, the evaluation unit 320, and the output unit 330 are examples of functional blocks of the processing device 305. All or some of the receiving unit 310, the evaluation unit 320, and the output unit 330 may be configured by hardware such as DSP, ASIC, PLD, or FPGA.
  • the receiving unit 310 receives an evaluation reproduction log (a reproduction log indicating a status regarding reproduction of a moving image on the information processing apparatus 20) from the information processing apparatus 20.
  • an evaluation reproduction log a reproduction log indicating a status regarding reproduction of a moving image on the information processing apparatus 20
  • the learning model 304a used by the evaluation unit 320 is a learned model that has learned relational data indicating the relationship between the situation regarding the reproduction of the moving image and the evaluation regarding the situation regarding the reproduction of the moving image.
  • the learning model 304a is defined by a plurality of coefficients K.
  • the plurality of coefficients K are specified by machine learning using teacher data, which is an example of relational data.
  • the evaluation unit 320 uses the learning model 304a to evaluate the situation regarding the reproduction of the moving image performed by the reproduction software in the development stage, based on the evaluation reproduction log received by the reception unit 310.
  • the learning model 304a defines, for example, a neural network, typically a deep neural network.
  • the evaluation unit 320 is a neural network realized by the processing device (an example of a computer) 305, or more specifically, a functional block.
  • the evaluation unit 320 generates the output B according to the input A.
  • the learning model 304a is used in a program (for example, a program module that constitutes artificial intelligence software) that causes the processing device 305 to execute an operation that specifies the output A from the input A.
  • the learning model 304a includes a plurality of coefficients K applied to the calculation.
  • a predetermined response function is used for the calculation.
  • the multiple coefficients K are optimized by prior machine learning (deep learning) using relational data including multiple teacher data.
  • the input A and the output B are associated with each other. That is, the learning model 304a is a statistical model in which the relational data indicating the relation between the input A and the output B is learned.
  • the processing device 305 executes a calculation that applies a plurality of coefficients K and a predetermined response function to an unknown input A, and thereby tends to be extracted from a plurality of teacher data (between the input A and the output B).
  • Output B based on the relation (1), that is, an output B that is valid for the input A is generated.
  • a “playback log” (information indicating the status regarding playback of a moving image) is used as the input A, and an “evaluation” is used as the output B.
  • the evaluation unit 320 may be implemented by a neural network processor such as a Tensor Processing Unit (tensor processing unit) and a Neural Engine (neural engine), for example.
  • a neural network processor such as a Tensor Processing Unit (tensor processing unit) and a Neural Engine (neural engine), for example.
  • the output unit 330 outputs information based on the evaluation result by the evaluation unit 320 for the reproduction software at the development stage. For example, the output unit 330 outputs image information based on the evaluation result and audio information based on the evaluation result as the information based on the evaluation result.
  • FIG. 2 is a diagram showing an example of a learning model generation device 50 that generates a learning model 304a defined by a plurality of coefficients K.
  • elements that are the same as or similar to the elements described in FIG. 1 are assigned the same reference numerals and detailed description thereof is omitted.
  • the learning model generation device 50 generates teacher data for the learning model 304a and a plurality of coefficients K that define the learning model 304a.
  • the teacher data for the learning model 304a may be generated by a device different from the learning model generation device 50.
  • the learning model generation device 50 communicates with the information processing devices 41 to 4n (n is an integer of 1 or more) via the network NW.
  • the information processing devices 41 to 4n realize the reproduction executing unit 210 by executing the reproduction software which has been developed, not the reproduction software at the development stage, and the reproduction information for reproducing the moving image and the sound.
  • the configuration is the same as that of the information processing device 20 except that the added reproduction log is output.
  • each of the information processing devices 41 to 4n is referred to as an information processing device 4m (m is an integer of 1 to n).
  • the reproduction information is used to reproduce the moving image and sound reproduced by the information processing device 4m.
  • the reproduction information includes identification information of the distribution data used for reproducing the sound and the moving image in the information processing device 4m.
  • the reproduction information further includes, for each index number of the image frame forming the moving image, the image quality (resolution) of the image frame with the index number, whether or not the image frame is reproduced, and the audio to be synchronized with the image frame. The time difference between the reproduction timing and the image frame is shown.
  • the learning model generation device 50 is configured by a computer system including an input device 501, a display device 502, a speaker 503, a communication device 504, a storage device 505, and a processing device 506. Each element of the learning model generation device 50 is mutually connected by one or two or more buses. Further, each element of the learning model generation device 50 is configured by one or two or more devices.
  • the input device 501 is a device for inputting information used by the learning model generation device 50 to the learning model generation device 50.
  • the input device 501 receives user input such as instructions or questions from the user.
  • the input device 501 is, for example, a keyboard.
  • the input device 501 is not limited to the keyboard, and may be a touch panel, a mouse, a switch, a button, or the like that detects contact with the display surface of the display device 502.
  • the display device 502 displays various images under the control of the processing device 506. For example, the display device 502 displays a moving image based on the moving image data included in the distribution data.
  • the display device 502 is, for example, a liquid crystal display panel or an organic EL display panel.
  • the speaker 503 outputs various sounds by being controlled by the processing device 506. For example, the speaker 503 outputs a sound based on the sound data included in the distribution data.
  • the communication device 504 communicates with other devices, for example, the information processing devices 41 to 4n, via the network NW.
  • the communication device 504 is also called, for example, a network device, a network controller, a network card, or a communication module.
  • the storage device 505 is a recording medium that can be read by the processing device 506.
  • the storage device 505 stores a control program executed by the processing device 506 and various data used by the processing device 506.
  • the storage device 505 is composed of, for example, a non-volatile memory (for example, ROM, EPROM, and EEPROM) and a volatile memory (for example, RAM).
  • the processing device 506 is a processor that controls the learning model generation device 50.
  • the processing device 506 includes, for example, one or more chips.
  • the processing device 506 is configured by a central processing unit (CPU).
  • CPU central processing unit
  • Some or all of the functions of the processing device 506 may be realized by hardware such as DSP, ASIC, PLD, or FPGA.
  • the processing device 506 executes various processes in parallel or sequentially.
  • the processing device 506 reads the control program from the storage device 505.
  • the processing device 506 functions as the receiving unit 510, the reproducing unit 520, the evaluation obtaining unit 530, the teacher data generating unit 540, and the learning model generating unit 550 by executing the control program.
  • the reception unit 510, the reproduction unit 520, the evaluation acquisition unit 530, the teacher data generation unit 540, and the learning model generation unit 550 are examples of functional blocks of the processing device 506. All or some of the receiving unit 510, the reproducing unit 520, the evaluation obtaining unit 530, the teacher data generating unit 540, and the learning model generating unit 550 may be configured by hardware such as DSP, ASIC, PLD, or FPGA.
  • the receiving unit 510 receives the reproduction log from the information processing device 4m via the communication device 504.
  • the reproduction log received from the information processing device 4m will be referred to as a "learning reproduction log”.
  • the reproduction log for learning shows the number of consecutive frames dropped and the time difference between the reproduction timings of the audio and the moving image. Replay information is added to the learning log for learning.
  • the reproduction unit 520 reproduces the reproduction status of the moving image and the audio in the information processing device 4m, using the display device 502 and the speaker 503, based on the reproduction information for each reproduction reproduction log for learning.
  • the evaluator who evaluates the reproduction quality of the moving image evaluates the quality of the reproduction of the moving image reproduced using the display device 502 and the speaker 503.
  • the evaluator inputs the evaluation result of the reproduction quality of the moving image into the input device 501.
  • the evaluation acquisition unit 530 receives, from the input device 501, the evaluation result regarding the reproduction quality of the moving image for each learning reproduction log.
  • the teacher data generation unit 540 generates teacher data for each learning reproduction log.
  • Each teacher data is a set of a learning reproduction log and a quality evaluation result (label) regarding reproduction of a moving image reproduced based on the learning reproduction log.
  • the learning model generation unit 550 generates a plurality of coefficients K defining the learning model 304a by performing machine learning (deep learning) using the teacher data generated by the teacher data generation unit 540.
  • FIG. 3 is a flowchart for explaining an example of the operation of the learning model generation device 50.
  • reference to the communication devices 204 and 504 will be omitted in the communication operation between the learning model generation device 50 and the information processing device 4m.
  • the receiving unit 510 receives the learning reproduction log from the information processing device 4m (step S100).
  • the receiving unit 510 receives a learning reproduction log that the information processing device 4m voluntarily transmits.
  • the receiving unit 510 may transmit a request for a learning reproduction log to the information processing apparatus 4m, and may receive a learning reproduction log transmitted by the information processing apparatus 4m in response to the learning reproduction log request.
  • the reproduction unit 520 reproduces the reproduction status of the moving image and the audio in the information processing device 4m based on the reproduction information for each learning reproduction log (step S102).
  • the reproduction unit 520 first obtains from the distribution server 10 the distribution data specified by the identification information of the distribution data indicated by the reproduction information. Specifically, the reproduction unit 520 acquires, from the distribution server 10, distribution data indicating a moving image having the highest image quality among the image quality indicated for each index number of the image frame in the reproduction information. The reproduction unit 520 may obtain the distribution data from a server different from the distribution server 10. Subsequently, the reproduction unit 520 reproduces the reproduction state of the moving image and the audio indicated by the reproduction information using the display device 502 and the speaker 503 based on the distribution data and the reproduction information.
  • the image quality of each image frame, the presence or absence of reproduction of each image frame, and the time difference between the reproduction timing of the sound of each image frame and the sound are reproduced. Since the distribution data indicating the moving image having the highest image quality among the image quality indicated for each image frame index number is used, the conversion to any image quality indicated for each image frame index number is possible.
  • the evaluator evaluates the reproduction quality of the moving image reproduced using the display device 502 and the speaker 503, and inputs the evaluation result to the input device 501.
  • the evaluation acquisition unit 530 receives the evaluation result input to the input device 501 (step S104).
  • the teacher data generation unit 540 deletes the reproduction information from the reproduction reproduction log for learning.
  • the teacher data generation unit 540 evaluates the learning reproduction log for each learning reproduction log in which the reproduction information is deleted, and the reproduction quality of the moving image reproduced based on the learning reproduction log.
  • a pair of the result and is generated as teacher data step S106. Therefore, the teacher data indicates the relationship between the situation regarding the reproduction of the moving image (the situation indicated by the reproduction log) and the evaluation regarding the situation regarding the reproduction of the moving image (evaluation result).
  • the learning model generation unit 550 performs machine learning (deep learning) using the teacher data generated by the teacher data generation unit 540 (step S108), and thereby a plurality of coefficients K defining the learning model 304a. Is specified (step S110).
  • the plurality of coefficients K specified by the learning model generation unit 550 are stored in the storage device 304 shown in FIG. 1 by the evaluator, for example.
  • the evaluation device 30 implements the evaluation unit 320 based on the plurality of coefficients K stored in the storage device 304.
  • the learning model 304a learns the relationship between the situation regarding the reproduction of the moving image and the evaluation regarding the situation regarding the reproduction of the moving image.
  • FIG. 4 is a flowchart for explaining an example of the operation of the evaluation device 30.
  • the reference to the communication devices 204 and 301 in the communication between the evaluation device 30 and the information processing device 20 will be omitted.
  • the receiving unit 310 receives the reproduction log for evaluation from the information processing device 20 (step S200).
  • step S200 the receiving unit 310 receives the reproduction log for evaluation that the information processing apparatus 20 voluntarily transmits.
  • the receiving unit 310 may send a request for a reproduction log for evaluation to the information processing apparatus 20, and receive the reproduction log for evaluation transmitted by the information processing apparatus 20 in response to the request for the reproduction log for evaluation.
  • the evaluation unit 320 uses the learning model 304a to evaluate the situation regarding the reproduction of the moving image performed by the reproduction software in the development stage based on the reproduction log received by the reception unit 310 (step S202).
  • step S202 the evaluation unit 320 receives the evaluation reproduction log as an input, and outputs the evaluation of the situation regarding the reproduction of the moving image performed by the reproduction software in the development stage.
  • the receiving unit 310 may input the reproduction log for evaluation to the evaluation unit 320.
  • the output unit 330 outputs information based on the result of the evaluation performed by the evaluation unit 320 on the reproduction software at the development stage (step S204).
  • step S204 the output unit 330 generates evaluation image information indicating the result of the evaluation performed by the evaluation unit 320, and outputs the evaluation image information to the display device 302.
  • the display device 302 Upon receiving the evaluation image information, the display device 302 displays the image indicated by the evaluation image information (the image indicating the result of the evaluation performed by the evaluation unit 320).
  • the output unit 330 indicates by the first image that the reproduction software performed in the development stage has poor reproduction quality.
  • the first evaluation image information is output to the display device 302.
  • the first image represents, for example, a character "reproduction quality is poor.” Note that the first image is not limited to the image representing the character "reproduction quality is poor.”, but may be an image representing the symbol "x", for example, and can be appropriately changed.
  • the display device 302 Upon receiving the first evaluation image information, the display device 302 displays the first image.
  • the output unit 330 uses the second evaluation image that indicates that the quality of reproduction performed by the reproduction software in the development stage is good by the second image.
  • the information is output to the display device 302.
  • the second image represents, for example, a character "The reproduction quality is good.” It should be noted that the second image is not limited to the image showing the character “The reproduction quality is good.”, but may be an image showing the symbol “ ⁇ ”, for example, and can be changed appropriately.
  • the display device 302 displays the second image.
  • step S204 the output unit 330 generates evaluation voice information based on the result of the evaluation performed by the evaluation unit 320, and outputs the evaluation voice information to the speaker 303.
  • the speaker 303 Upon receiving the evaluation voice information, the speaker 303 outputs a sound according to the evaluation voice information.
  • the output unit 330 expresses by the first voice that the reproduction quality performed by the reproduction software in the development stage is poor.
  • the first evaluation voice information to be output is output to the speaker 303.
  • the first voice is, for example, a voice "reproduction quality is poor.”
  • the first voice is not limited to the voice "reproduction quality is poor.”
  • the speaker 303 Upon receiving the first evaluation voice information, the speaker 303 outputs the first voice according to the first evaluation voice information.
  • the output unit 330 expresses by the second voice that the reproduction quality performed by the reproduction software in the development stage is good.
  • the evaluation voice information is output to the speaker 303.
  • the second sound is, for example, a sound "the reproduction quality is good.”
  • the second voice is not limited to the voice "The reproduction quality is good", but may be the voice "OK", for example, and can be changed as appropriate.
  • the speaker 303 Upon receiving the second evaluation audio information, the speaker 303 outputs the second audio corresponding to the second evaluation audio information.
  • the output unit 330 may output an electronic file indicating the evaluation result to another device as information based on the result of the evaluation performed by the evaluation unit 320.
  • the electronic file showing the evaluation result for example, an electronic file of text data showing the evaluation result and an electronic file for spreadsheet showing the evaluation result are used.
  • the evaluation unit 320 uses the learning model 304a that learned the relational data indicating the relationship between the situation regarding the reproduction of the moving image and the evaluation for the situation, and the reproduction situation regarding the reproduction of the moving image performed by the reproduction software. Evaluate. For this reason, compared to a case where a person evaluates the reproduction state of a moving image performed by the reproduction software in the development stage, the labor of the test in the development stage of the reproduction software can be reduced.
  • the receiving unit 310 receives a playback log for evaluation indicating the status regarding playback of a moving image performed by the playback software, and the evaluation unit 320 displays the evaluation result according to the playback status indicated by the playback log for evaluation received by the receiving unit 310. To generate. Therefore, there is an advantage that the evaluation device 30 does not need to generate the reproduction log for evaluation.
  • the status regarding the playback of the moving picture includes both the status of the number of unplayed image frames and the status of the timing difference between the playback timing of the audio and the moving picture.
  • the higher the number of non-reproduced image frames the worse the reproduction quality of the moving image.
  • the longer the time difference between the playback timings of audio and video the worse the quality of video playback.
  • a learning model is used that learns the relationship between the situation of the number of unreproduced image frames and the situation of the reproduction of the video that does not include the situation of the reproduction timing of the audio and the video, and the evaluation thereof.
  • the reliability of the evaluation performed by the evaluation unit 320 can be increased as compared with the evaluation.
  • the status related to video playback includes one of the status of the number of non-playback image frames and the status of timing difference between audio and video playback, status related to video playback that does not include either
  • the reliability of the evaluation performed by the evaluation unit 320 can be made higher than the evaluation using the learning model in which the relationship between the evaluation and the evaluation is learned.
  • the status of the number of non-playback image frames indicates the number of consecutive non-playback image frames.
  • the situation regarding the reproduction of the moving image indicated by the reproduction log may further include the state of the load of the device that is reproducing the moving image.
  • the load status of a device playing a moving image (hereinafter referred to as “playback device”) is also referred to as “device load status”.
  • the load status of the device indicates the decoding time required for the playback device to decode the distribution data (hereinafter, simply referred to as “decoding time”). Generally, since the decoding time of moving image data is longer than the decoding time of audio data, the time required for decoding moving image data is used as the decoding time included in the load status of the device.
  • the device load status further indicates a decryption time (hereinafter, referred to as “decrypt time”) required for decryption, which is a reproduction device decrypting distribution data encryption (for example, encryption using DRM).
  • decrypt time a decryption time required for decryption
  • the encryption of distribution data is not limited to the encryption using DRM.
  • the decryption time of moving image data is longer than the decryption time of audio data, the decryption time of moving image data is used as the decryption time included in the load status of the device.
  • the load status of the device may include only one of the decoding time and the decryption time, not both.
  • the status regarding the playback of the video indicated by the playback log may further indicate the bit rate of the video data.
  • each of the learning reproduction log and the evaluation reproduction log includes the number of consecutive frames dropped, the time difference between the reproduction timings of audio and video, the bit rate of video data, and the decoding time. And the decryption time.
  • the longer the decoding time the more likely the number of consecutive frames dropped.
  • the longer the decoding time the longer the time lag between the audio and video playback timings tends to be. Therefore, the longer the decoding time, the worse the reproduction quality of the moving image.
  • the longer the decryption time the more likely the number of consecutive frames dropped.
  • the longer the decryption time the longer the time difference between the audio and video playback timing. Therefore, the longer the decryption time, the worse the reproduction quality of the moving image.
  • the number of consecutive dropped frames, the time difference between audio and video playback timing, the video data bit rate, the decoding time, and the decryption time affect the video playback quality.
  • the learning model 304a has a relationship between the evaluation and the reproduction log indicating the number of consecutive frames dropped, the time difference between the audio and video reproduction timing, the video data bit rate, the decoding time, and the decryption time.
  • the learning model 304a of the first embodiment learns relational data indicating the relation between the evaluation and the reproduction log indicating only the number of continuous frame dropouts and/or the time difference between the reproduction timings of audio and moving images. .. Therefore, the result of the evaluation performed by the evaluation unit 320 in the first modification has higher reliability than the result of the evaluation performed by the evaluation unit 320 in the first embodiment.
  • the playback log for learning and the playback log for evaluation show only the number of consecutive frames dropped, the time difference between the playback timing of audio and video, the bit rate of video data, the decoding time, and the decryption time, but not all. May be.
  • the playback log for learning and the playback log for evaluation include a decoding time and a bit rate of video data in addition to one or both of the number of consecutive dropped frames and the time difference between the audio and video playback timings.
  • Decryption time only one or two may be shown.
  • the playback log for learning and the playback log for evaluation include the decoding time and the bit rate of the video data in addition to the number of consecutive frames dropped and the time difference between the audio and video playback timing. , And the decryption time may all be shown.
  • the learning model that learned the relationship between the situation of the number of unreproduced image frames and the situation of the reproduction of the video that does not include the situation of the timing difference between the reproduction timing of the audio and the video and the evaluation thereof.
  • the reliability of the evaluation performed by the evaluation unit 320 can be increased as compared with the evaluation using.
  • the evaluation device 30 illustrated in FIG. 1 may include the function of the learning model generation device 50 illustrated in FIG. 2.
  • FIG. 5 is a diagram showing an example of the evaluation device 30 including the function of the learning model generation device 50. 5, elements that are the same as or similar to the elements described in FIG. 1 or FIG. 2 are given the same reference numerals, and detailed description thereof will be omitted.
  • the receiving unit 310 also serves as the receiving unit 510.
  • the communication device 301 also serves as the communication device 504.
  • the display device 302 also serves as the display device 502.
  • the speaker 303 also serves as the speaker 503.
  • the storage device 304 also serves as the storage device 505.
  • the processing device 305 also serves as the processing device 506.
  • the processing device 305 reads the control program and the learning model 304a stored in the storage device 304.
  • the processing device 305 executes the control program and the learning model 304a to obtain the receiving unit 310, the evaluation unit 320, the output unit 330, the reproduction unit 520, the evaluation acquisition unit 530, the teacher data generation unit 540, and the learning model generation unit 550. Function as.
  • the learning model generation unit 550 can store the plurality of coefficients K specified by the learning model generation unit 550 in the storage device 304. Therefore, a person (for example, an evaluator) can save the trouble of storing the plurality of coefficients K specified by the learning model generation unit 550 in the storage device 304.
  • the evaluation device 30 may further include a vibration device.
  • the output unit 330 may vibrate the vibrating device using the vibration information based on the evaluation result performed by the evaluation unit 320. For example, when the result of the evaluation performed by the evaluation unit 320 indicates that the reproduction quality is poor, the output unit 330 outputs the vibration information that causes the vibration device to vibrate, to the vibration device. On the other hand, when the result of the evaluation performed by the evaluation unit 320 indicates that the quality of reproduction is good, the output unit 330 does not output the vibration information for vibrating the vibration device to the vibration device.
  • first storage device that stores the reproduction log for evaluation in the information processing device 20
  • the reproduction log for evaluation of the information processing device 20 may be received from the first storage device.
  • the receiving unit 310 receives the reproduction log for evaluation that the first storage device voluntarily transmits.
  • the receiving unit 310 may send a request for a playback log for evaluation to the first storage device, and may receive a playback log for evaluation sent by the first storage device in response to the request for a playback log for evaluation. ..
  • the receiving unit 510 when there is a device (hereinafter, referred to as “second storage device”) that stores the reproduction log for learning in the information processing device 4m, the receiving unit 510 The reproduction log for learning of the information processing device 4m may be received from the second storage device.
  • the receiving unit 310 that also serves as the receiving unit 510 may receive the learning reproduction log of the information processing device 4m from the second storage device.
  • the receiving unit 510 receives the learning reproduction log voluntarily transmitted by the second storage device.
  • the receiving unit 310 may transmit a request for a learning reproduction log to the second storage device, and may receive a learning reproduction log that the second storage device transmits in response to the request for the learning reproduction log. ..
  • the evaluation device 30 may be installed with reproduction software at the development stage.
  • the processing device 305 may read the reproduction software at the development stage from the evaluation device 30.
  • the processing device 305 may further function as the reproduction executing unit 210 by executing the reproduction software at the development stage read from the evaluation device 30. In this case, since the evaluation device 30 can generate the reproduction log for evaluation, the information processing device 20 can be omitted.
  • the learning model 304a may be represented by, for example, an SVM (Support Vector Machine) or an HMM (Hidden Markov Model).
  • the storage devices 102, 205, 304, and 505 are flexible disks, magneto-optical disks (for example, compact disks, digital versatile disks, Blu-ray (registered trademark) disk, smart card, flash memory device (for example, card, stick, key drive), CD-ROM (Compact Disc-ROM), register, removable disk, hard disk, floppy (registered trademark) disk , Magnetic strips, databases, servers and other suitable storage media.
  • the program may be transmitted from the network via an electric communication line.
  • the first embodiment and each of the first to seventh modifications are LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G, 5G, and FRA ( Future Radio Access), W-CDMA (registered trademark), GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802.20. , UWB (Ultra-WideBand), Bluetooth (registered trademark), and other suitable systems, and/or may be applied to a next-generation system extended based on these systems.
  • each of the first embodiment and each of the first modification to the seventh modification may be represented using any of various different technologies.
  • data, information, bits, chips, etc. that may be referred to throughout the above description may be represented by voltage, current, electromagnetic waves, magnetic fields, magnetic particles, optical fields, photons, or any combination thereof. Good. Note that the terms described in the present specification and/or the terms necessary for understanding the present specification may be replaced with terms having the same or similar meanings.
  • the input/output information and the like may be stored in a specific location (for example, a memory) or may be managed by a management table. Good. Information that is input/output may be overwritten, updated, or added. The output information and the like may be deleted. The input information and the like may be transmitted to another device.
  • the determination may be made based on the value (0 or 1) represented by 1 bit, or the true/false value. It may be performed based on (Boolean: true or false) or may be performed based on comparison of numerical values (for example, comparison with a predetermined value).
  • Each function illustrated in FIG. 1, FIG. 2 or FIG. 5 is realized by an arbitrary combination of hardware and software. Further, each function may be realized by a single device, or may be realized by two or more devices that are configured separately from each other.
  • the software may use a wired technology such as coaxial cable, fiber optic cable, twisted pair and digital subscriber line (DSL) and/or wireless technology such as infrared, wireless and microwave to websites, servers, or other When transmitted from a remote source, these wireline and/or wireless technologies are included within the definition of transmission medium.
  • a wired technology such as coaxial cable, fiber optic cable, twisted pair and digital subscriber line (DSL) and/or wireless technology such as infrared, wireless and microwave to websites, servers, or other
  • DSL digital subscriber line
  • wireless technology such as infrared, wireless and microwave
  • the information processing devices 20 and 41 to 4n may be mobile stations.
  • Mobile stations are defined by those skilled in the art as subscriber stations, mobile units, subscriber units, wireless units, remote units, mobile devices, wireless devices, wireless communication devices, remote devices, mobile subscriber stations, access terminals, mobile terminals, wireless. It may also be referred to as a terminal, remote terminal, handset, user agent, mobile client, client, or some other suitable term.

Abstract

La présente invention concerne un dispositif d'évaluation comprenant : une unité d'évaluation qui évalue un état de reproduction lié à une reproduction vidéo effectuée par un logiciel de reproduction vidéo, une telle évaluation utilisant un modèle d'apprentissage dans lequel des données de relation apprises indiquent la relation entre un état lié à la reproduction vidéo et une évaluation d'un tel état ; et une unité de sortie qui délivre des informations sur la base des résultats d'évaluation provenant de l'unité d'évaluation.
PCT/JP2019/043656 2019-02-01 2019-11-07 Dispositif d'évaluation WO2020158095A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019-016762 2019-02-01
JP2019016762 2019-02-01

Publications (1)

Publication Number Publication Date
WO2020158095A1 true WO2020158095A1 (fr) 2020-08-06

Family

ID=71841986

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2019/043656 WO2020158095A1 (fr) 2019-02-01 2019-11-07 Dispositif d'évaluation

Country Status (1)

Country Link
WO (1) WO2020158095A1 (fr)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH08163559A (ja) * 1994-12-07 1996-06-21 Graphics Commun Lab:Kk 低遅延モード画像復号方法および装置
JPH1169325A (ja) * 1997-08-11 1999-03-09 Ando Electric Co Ltd 動画通信管理装置
WO2007111208A1 (fr) * 2006-03-24 2007-10-04 Matsushita Electric Industrial Co., Ltd. Dispositif de reproduction, dispositif de débogage, système lsi, et programme correspondant
US20080037864A1 (en) * 2006-08-08 2008-02-14 Chunhong Zhang System and method for video quality measurement based on packet metric and image metric
JP2008538642A (ja) * 2005-04-22 2008-10-30 マイクロソフト コーポレーション 多面的システムの有能性分析
JP2013537748A (ja) * 2010-07-30 2013-10-03 トムソン ライセンシング ビデオ品質を測定する方法および装置
JP2018522448A (ja) * 2015-05-11 2018-08-09 ネットフリックス・インコーポレイテッドＮｅｔｆｌｉｘ， Ｉｎｃ． 知覚的ビデオ品質を予測する技術

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH08163559A (ja) * 1994-12-07 1996-06-21 Graphics Commun Lab:Kk 低遅延モード画像復号方法および装置
JPH1169325A (ja) * 1997-08-11 1999-03-09 Ando Electric Co Ltd 動画通信管理装置
JP2008538642A (ja) * 2005-04-22 2008-10-30 マイクロソフト コーポレーション 多面的システムの有能性分析
WO2007111208A1 (fr) * 2006-03-24 2007-10-04 Matsushita Electric Industrial Co., Ltd. Dispositif de reproduction, dispositif de débogage, système lsi, et programme correspondant
US20080037864A1 (en) * 2006-08-08 2008-02-14 Chunhong Zhang System and method for video quality measurement based on packet metric and image metric
JP2013537748A (ja) * 2010-07-30 2013-10-03 トムソン ライセンシング ビデオ品質を測定する方法および装置
JP2018522448A (ja) * 2015-05-11 2018-08-09 ネットフリックス・インコーポレイテッドＮｅｔｆｌｉｘ， Ｉｎｃ． 知覚的ビデオ品質を予測する技術

Similar Documents

Publication Publication Date Title
US11842036B2 (en) Remotely configuring windows displayed on a display device
US8788544B2 (en) Method and apparatus for managing data
JP4883342B2 (ja) 情報処理装置および方法、並びにプログラム
US10455286B2 (en) Protected media decoding system supporting metadata
CN102662919A (zh) 对内容片段设置书签
CN103167334A (zh) 用于经由hdmi信号的广播接收机双显示的便携式终端的装置和方法
CN110149539A (zh) 多媒体文件播放方法、装置、电子设备和存储介质
JP2014523692A (ja) 最適な表示のための、トランスコーディング検出およびコンテンツの調整
JPWO2016129216A1 (ja) 再生装置および出力装置
JP2014075094A (ja) 表示装置、表示システム、携帯情報端末および表示装置の制御方法
US20160359932A1 (en) Display device and method of controlling the same
JP2016005268A (ja) 情報伝送システム、情報伝送方法、及びプログラム
JP2018060581A (ja) 情報処理装置、情報処理方法およびプログラム
KR20110052817A (ko) 외부기기로의 컨텐츠 제공방법
WO2020158095A1 (fr) Dispositif d'évaluation
JP2015050631A (ja) 表示システム、情報端末、選択制御プログラム及び選択制御方法
JP5418118B2 (ja) 電子機器
US20230126176A1 (en) System and method for selection and transmission of personalized content tracks
WO2020158093A1 (fr) Dispositif de commande et dispositif de communication
US20060218585A1 (en) Edited contents reproducing method and apparatus, and computer-readable storage medium
KR20180010955A (ko) 전자 장치 및 그의 제어 방법
JP2013106285A (ja) 携帯端末装置、暗号鍵生成方法、およびプログラム。
JP5306043B2 (ja) デジタルｔｖ放送受信システム、デジタルｔｖ放送受信処理装置
JP7369604B2 (ja) 情報処理システム
JP5416599B2 (ja) 動画作成装置及び動画作成方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19914011

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19914011

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP