CN116347060A

CN116347060A - Camera performance evaluation method, device, readable storage medium and system

Info

Publication number: CN116347060A
Application number: CN202111602591.6A
Authority: CN
Inventors: 周至捷
Original assignee: Beijing Xiaomi Mobile Software Co Ltd
Current assignee: Beijing Xiaomi Mobile Software Co Ltd
Priority date: 2021-12-24
Filing date: 2021-12-24
Publication date: 2023-06-27

Abstract

The disclosure relates to a performance evaluation method, a device, a readable storage medium and a system, which are used for automatically evaluating the performance of a camera and improving the efficiency of evaluating the performance of the camera. The method comprises the following steps: in response to receiving an evaluation instruction for camera performance, controlling a camera to be evaluated to execute a test case related to the camera to be evaluated, wherein the test case is used for describing the operation of the camera to be evaluated; monitoring the execution process of the test case to obtain performance parameters of the camera to be evaluated, which correspond to the test case; and evaluating the performance of the camera to be evaluated according to the performance parameters of the camera to be evaluated, which correspond to the test case.

Description

Camera performance evaluation method, device, readable storage medium and system

Technical Field

The disclosure relates to the technical field of product testing, and in particular relates to a performance evaluation method, a performance evaluation device, a readable storage medium and a performance evaluation system.

Background

With the development of science and technology, various terminal devices are widely used in life and work. Among them, mobile terminals (e.g., mobile phones with cameras) are becoming more popular, and people have higher requirements for camera performance, so improving camera performance is becoming a problem that mobile phone manufacturers need to solve. In order to provide better use experience for users, not only the definition of photographing by the camera needs to be improved, but also the smoothness of the use of the camera by the users needs to be ensured, so that the performance of the camera needs to be tested to ensure the use experience of the users by the cameras.

Disclosure of Invention

To overcome the problems in the related art, the present disclosure provides a performance evaluation method, apparatus, readable storage medium, and system.

According to a first aspect of embodiments of the present disclosure, there is provided a camera performance evaluation method, including:

in response to receiving an evaluation instruction for camera performance, controlling a camera to be evaluated to execute a test case related to the camera to be evaluated, wherein the test case is used for describing the operation of the camera to be evaluated;

monitoring the execution process of the test case to obtain performance parameters of the camera to be evaluated, which correspond to the test case;

and evaluating the performance of the camera to be evaluated according to the performance parameters of the camera to be evaluated, which correspond to the test case.

Optionally, the performance parameter is smoothness of the camera; the monitoring the execution process of the test case, obtaining the performance parameters of the camera to be evaluated, which correspond to the test case, includes:

monitoring the execution process of the test case, and obtaining the duration of the operation corresponding to the test case executed by the camera to be evaluated;

acquiring a fluency computation function corresponding to a scene type to which the test case belongs, wherein the scene type comprises a starting type, a working mode switching type, a video recording type and a photographing type;

And determining the fluency of the camera to be evaluated, which corresponds to the test case, according to the fluency calculation function and the duration.

Optionally, the test cases are multiple; the evaluating the performance of the camera to be evaluated according to the performance parameters of the camera to be evaluated corresponding to the test case includes:

for each test case, determining a value weight value of the test case according to the use frequency and/or the attention degree of the operation corresponding to the test case, wherein the value weight value is used for representing the importance degree of the operation corresponding to the test case;

determining the fluency of the camera to be evaluated according to the value weight value of each test case and the fluency of the camera to be evaluated corresponding to each test case;

and evaluating the performance of the camera to be evaluated according to the fluency of the camera to be evaluated.

Optionally, the controlling the camera to be evaluated to execute the test case related to the camera to be evaluated in response to receiving the evaluation instruction for the performance of the camera includes:

in response to receiving an evaluation instruction for camera performance, controlling the camera to be evaluated to execute N times on the same test case, wherein N is an integer greater than or equal to 2;

The monitoring the execution process of the test case, and obtaining the duration of the operation corresponding to the test case executed by the camera to be evaluated, includes:

and monitoring the N execution processes of the test case, obtaining N time durations for the camera to be evaluated to execute the operation corresponding to the test case, and determining the average value of the N time durations as the time duration for the camera to be evaluated to execute the operation corresponding to the test case.

Optionally, the monitoring the execution process of the test case, to obtain the performance parameter of the camera to be evaluated, which corresponds to the test case, includes:

acquiring multi-frame images of the camera to be evaluated for executing the test case;

and obtaining performance parameters of the camera to be evaluated, which correspond to the test case, according to the multi-frame images.

Optionally, the acquiring the multi-frame image when the camera to be evaluated executes the test case includes:

when the camera to be evaluated executes the test case, controlling an industrial camera to acquire multi-frame images of the camera to be evaluated executing the test case;

and acquiring multi-frame images of the test case executed by the camera to be evaluated from the industrial camera.

According to a second aspect of the embodiments of the present disclosure, there is provided a camera performance evaluation apparatus including:

a control module configured to control a camera to be evaluated to execute a test case related to the camera to be evaluated in response to receiving an evaluation instruction for camera performance, the test case being used for describing an operation on the camera to be evaluated;

the acquisition module is configured to monitor the execution process of the test case and acquire performance parameters of the camera to be evaluated, which correspond to the test case;

and the evaluation module is configured to evaluate the performance of the camera to be evaluated according to the performance parameters of the camera to be evaluated, which correspond to the test cases.

Optionally, the performance parameter is smoothness of the camera; the acquisition module comprises:

the first acquisition sub-module is configured to monitor the execution process of the test case and acquire the duration of the operation corresponding to the test case executed by the camera to be evaluated;

the second acquisition submodule is configured to acquire a fluency computation function corresponding to a scene type to which the test case belongs, wherein the scene type comprises a starting type, a working mode switching type, a video recording type and a photographing type;

And the first determination submodule is configured to determine the fluency of the camera to be evaluated, which corresponds to the test case, according to the fluency calculation function and the duration.

Optionally, the test cases are multiple; the evaluation module includes:

the second determining submodule is configured to determine a value weight value of each test case according to the use frequency and/or the attention degree of the operation corresponding to the test case, wherein the value weight value is used for representing the importance degree of the operation corresponding to the test case;

the third determining submodule is configured to determine the fluency of the camera to be evaluated according to the value weight value of each test case and the fluency of the camera to be evaluated corresponding to each test case;

and the evaluation sub-module is configured to evaluate the performance of the camera to be evaluated according to the fluency of the camera to be evaluated.

Optionally, the control module is configured to: in response to receiving an evaluation instruction for camera performance, controlling the camera to be evaluated to execute N times on the same test case, wherein N is an integer greater than or equal to 2;

the first acquisition sub-module is configured to: and monitoring the N execution processes of the test case, obtaining N time durations for the camera to be evaluated to execute the operation corresponding to the test case, and determining the average value of the N time durations as the time duration for the camera to be evaluated to execute the operation corresponding to the test case.

Optionally, the acquiring module includes:

a third obtaining sub-module configured to obtain multi-frame images of the test case executed by the camera to be evaluated;

and the fourth acquisition submodule is configured to acquire performance parameters of the camera to be evaluated, which correspond to the test case, according to the multi-frame images.

Optionally, the third obtaining submodule includes:

the control sub-module is configured to control the industrial camera to acquire multi-frame images of the camera to be evaluated for executing the test case when the camera to be evaluated executes the test case;

and a fifth acquisition sub-module configured to acquire multi-frame images of the test case executed by the camera to be evaluated from the industrial camera.

According to a third aspect of the embodiments of the present disclosure, there is provided a camera performance evaluation apparatus including:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to:

According to a fourth aspect of embodiments of the present disclosure, there is provided a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the steps of the camera performance evaluation method provided by the first aspect of the present disclosure.

According to a fifth aspect of embodiments of the present disclosure, there is provided a camera performance evaluation system, comprising: user equipment, detection equipment, industrial cameras;

the user equipment is provided with a camera to be evaluated and a script for automatically executing test cases, and the user equipment is used for calling the script under the control of the detection equipment so that the camera to be evaluated can execute the test cases related to the camera to be evaluated;

the industrial camera is used for acquiring multi-frame images of the test case executed by the camera to be evaluated under the control of the detection equipment;

the detection device is used for executing the camera performance evaluation method provided by the first aspect of the disclosure.

The technical scheme provided by the embodiment of the disclosure can comprise the following beneficial effects:

by adopting the technical scheme, the performance parameters of the camera to be evaluated, which correspond to the test cases, are obtained by monitoring the process of executing the test cases by the camera to be evaluated, and the camera to be evaluated can be further evaluated according to the performance parameters.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.

FIG. 1 is a block diagram illustrating a camera performance evaluation system according to an example embodiment.

Fig. 2 is a flowchart illustrating a camera performance evaluation method according to an exemplary embodiment.

FIG. 3 is a flow chart illustrating one method of obtaining performance parameters of a camera under evaluation corresponding to a test case, according to one exemplary embodiment.

FIG. 4 is a flow chart illustrating an evaluation of performance of a camera to be evaluated according to an exemplary embodiment.

Fig. 5 is a block diagram illustrating a camera performance evaluation apparatus according to an exemplary embodiment.

Fig. 6 is a block diagram illustrating a camera performance evaluation apparatus according to an exemplary embodiment.

Detailed Description

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the accompanying claims.

As background art, to ensure a user's experience with a camera, camera performance needs to be evaluated to determine if the camera performance is good. In the related art, subjective judgment or evaluation modes are often adopted to evaluate the performance of the camera, however, the subjective judgment or evaluation modes are difficult to fit with the actual use experience of a user, and the performance of the camera cannot be evaluated comprehensively and accurately, namely, the efficiency of evaluating the performance of the camera in the related art is lower. In addition, in the development stage of the camera, the accuracy of subjective judgment of the performance of the camera is low, so that the advantages and disadvantages of each version of the camera are difficult to objectively evaluate, and the camera cannot be accurately guided by technicians in the development stage of the performance optimization of the camera.

In view of this, the present disclosure provides a camera performance evaluation method, apparatus, readable storage medium, and system to improve accuracy of camera performance evaluation.

Before describing in detail the camera performance evaluation method provided by the present disclosure, a description will be first given of a camera performance evaluation system. FIG. 1 is a block diagram illustrating a camera performance evaluation system according to an example embodiment. As shown in fig. 1, the camera performance evaluation system may include: user device 10, detection device 20, and industrial camera 30. Wherein the detection device 20 may communicate with the user device 10, the industrial camera 30 via one or more networks.

In the present disclosure, the user device 10 and the detection device 20 may each include, but are not limited to, a palm top computer, a wearable computer device, a personal digital assistant, a tablet computer, a notebook computer, a desktop computer, a mobile phone, a smart phone.

The user device 10 is provided with a camera to be evaluated and a script for automatically executing test cases, and the user device 10 may be controlled by the detection device 20 to invoke the script so that the camera to be evaluated can execute the test cases related to the camera to be evaluated. The industrial camera 30 is used to acquire multi-frame images of test cases performed by the camera under evaluation under the control of the detection device 20. The detection device 20 is used to perform a camera performance evaluation method described below.

The value is stated that the test case executed by the camera to be evaluated may be used to describe the operation of the camera to capture images with different magnifications, in which case the camera performance evaluation system may further include a slide rail, where the user device 10 is disposed on the slide rail, and the slide rail may further drive the user device 10 to move, so that the camera to be evaluated disposed on the user device 10 may approach or depart from the object to be captured according to the test requirement, and further enable the camera to be evaluated to capture images with different magnifications. In addition, the test case performed by the camera under evaluation may be used to describe the operation of the camera to capture images of different ambient brightnesses, in which case the camera performance evaluation system may also include a light device to provide different ambient brightnesses to enable the camera under evaluation provided on the user device 10 to capture images of different brightnesses. The present disclosure is not particularly limited thereto.

Fig. 2 is a flowchart illustrating a camera performance evaluation method according to an exemplary embodiment, as shown in fig. 2, for the detection apparatus in fig. 1, including the following steps.

In step S21, in response to receiving the evaluation instruction for the camera performance, the camera to be evaluated is controlled to execute the test case related to the camera to be evaluated. The test cases are used for describing the operation of the camera to be evaluated.

In the present disclosure, the test cases related to the camera to be evaluated may be designed in advance by a technician. For example, a technician may collect historical data of a user using a camera to be evaluated through a crawler technology or automatically, obtain various operations of the user on the camera from the historical data, and then design test cases related to the camera to be evaluated according to the various operations. The test case may be used to describe a cold start operation, a hot start operation, an operation of switching a recording mode from a photographing mode to a video mode, an operation of a default photographing mode, an operation of a portrait photographing mode, an operation of ending video, and the like of the camera to be evaluated.

After receiving an evaluation instruction for the performance of the camera input by a technician, the detection device can control the camera to be evaluated to execute a test case related to the camera to be evaluated. For example, the detection device may communicate with the user device through the network to send a control instruction to the user device, where the control instruction is used to instruct the camera to be evaluated to execute the test case, so that the user device may invoke a script for automatically executing the test case after receiving the control instruction, so that the camera to be evaluated can execute the test case related to the camera to be evaluated.

In step S22, the execution process of the test case is monitored, and the performance parameters of the camera to be evaluated corresponding to the test case are obtained.

When the camera to be evaluated executes the test case, the detection equipment can monitor the execution process of the test case by itself or through other equipment, so as to obtain the performance parameters of the camera to be evaluated corresponding to the test case.

In step S23, the performance of the camera to be evaluated is evaluated according to the performance parameters of the camera to be evaluated corresponding to the test case.

The camera performance evaluation method provided by the present disclosure is described in detail below in a complete embodiment.

In one embodiment, the step S22 in fig. 2 of monitoring the execution process of the test case, and the specific implementation manner of obtaining the performance parameters of the camera to be evaluated, which correspond to the test case, may be: when the camera to be evaluated executes the test case, the detection equipment controls the user equipment where the camera to be evaluated is located to record the screen, and after the camera to be evaluated executes the test case, the performance parameters of the camera to be evaluated corresponding to the test case are obtained from the display screen picture recorded by the user equipment.

However, in general, the shooting speed is low in the screen recording process of the user equipment, which may leak to the complete process of executing the test case by the camera to be evaluated, so that the performance parameters of the camera to be evaluated corresponding to the test case cannot be accurately obtained. Therefore, in another embodiment, the step S22 in fig. 2 of monitoring the execution process of the test case, and the specific implementation manner of obtaining the performance parameters of the camera to be evaluated, which correspond to the test case, may be: and acquiring multi-frame images of the camera to be evaluated for executing the test case, and acquiring performance parameters of the camera to be evaluated corresponding to the test case according to the multi-frame images.

For example, when the camera to be evaluated executes the test case, the detection device may control the industrial camera to collect multiple frame images of the camera to be evaluated executing the test case, and obtain multiple frame images of the camera to be evaluated executing the test case from the industrial camera, so that the detection device may obtain performance parameters of the camera to be evaluated corresponding to the test case according to the multiple frame images.

In one embodiment, the performance parameter is smoothness of the camera. As shown in fig. 3, step S22 in fig. 2, which monitors the execution process of the test case, obtaining the performance parameters of the camera to be evaluated, which correspond to the test case, may further include steps S221 to S223.

In step S221, the execution process of the test case is monitored, and the duration of the operation corresponding to the test case executed by the camera to be evaluated is obtained.

In the disclosure, the duration of the operation corresponding to the test case of the camera to be evaluated can be determined according to the multi-frame image obtained by the industrial camera. The value indicates that the shutter time of the industrial camera is very short, the shooting speed is far higher than that of a general camera, and the object which moves fast can be shot, so that the detection equipment can acquire multi-frame images of the test case executed by the camera to be evaluated by using the industrial camera to determine the duration of the operation corresponding to the test case executed by the camera to be evaluated.

For example, the test case is used for describing the operation of the cold start camera, and when the execution process of the test case is monitored, the industrial camera can be controlled to acquire the image of the camera to be evaluated for executing the cold start operation. Thus, the multi-frame image collected by the industrial camera comprises the frame image of clicking the camera icon on the user equipment interface and the frame image of displaying the clear image on the user equipment. In one mode, the number of image frames separated from each other is determined, and the duration between the two images is determined according to the number of image frames and the frame length, wherein the duration is the duration of the cold start operation of the camera to be evaluated. In yet another manner, each frame of image acquired by the industrial camera is provided with a time stamp that characterizes the time at which the frame of image was acquired, so that the time period between the two frames of images can be determined from the time stamp on the frame of image that clicks the camera icon on the user device interface and the frame of image that displays a clear image on the user device.

As another example, the test case is used for describing a post-photographing operation, and when the camera to be evaluated executes the test case, the camera to be evaluated continuously clicks a photographing icon on the display interface of the user equipment to continuously photograph. The value indicates that the user equipment has the phenomenon of screen flashing when photographing, and the time interval between two screen flashes is the time for photographing a frame of image. Thus, in this example, the multi-frame image collected by the industrial camera may include a multi-frame screen shot image, and the detection device may determine, according to a time interval between two adjacent screen shots in the multi-frame image collected by the industrial camera, the time interval to perform the post-photographing operation.

As another example, the test case is used for describing the operation of ending the video recording, and when the camera to be evaluated executes the test case, the camera to be evaluated clicks the end video recording button (which may be a virtual button or an entity button) of the display interface of the user equipment to end the video recording. The values illustrate that when the camera ends recording, a thumbnail of the captured video is displayed on the display interface of the user device. Thus, in this example, the multi-frame image collected by the industrial camera includes an image of clicking the end video button and an image of displaying a thumbnail on the display interface of the user device, and the detection device may determine a time interval between the two frames of images collected by the industrial camera, and determine the time interval as a duration of performing an operation of ending video.

In the disclosure, for each test case, the camera to be evaluated may be controlled to execute the test case only once, so that the detection device obtains only one duration for each test case, and determines the duration as the duration of the operation corresponding to the test case executed by the camera to be evaluated. However, considering that the industrial camera sometimes cannot acquire each action of the camera to be evaluated to execute the test case, the detecting device cannot accurately acquire the duration of the operation corresponding to the test case executed by the camera to be evaluated.

Thus, in one embodiment, in response to receiving the evaluation instruction for the performance of the camera, step S21 in fig. 2 controls the camera to be evaluated to execute the test case related to the camera to be evaluated as follows: and in response to receiving the evaluation instruction for the camera performance, controlling the camera to be evaluated to execute N times on the same test case, wherein N is an integer greater than or equal to 2. That is, for each test case, the camera under evaluation is controlled to repeatedly execute N times. Accordingly, step S221 monitors the execution process of the test case, and the specific embodiment for obtaining the duration of the operation corresponding to the test case executed by the camera to be evaluated is as follows: and monitoring N execution processes of the test case, obtaining N time durations for the camera to be evaluated to execute the operation corresponding to the test case, and determining an average value of the N time durations as the time duration for the camera to be evaluated to execute the operation corresponding to the test case.

For example, assuming that the test case is used for describing the operation of the cold start camera, controlling the camera to be evaluated to execute the cold start operation N times, and in the process of executing the cold start operation once, the industrial camera can continuously take pictures of the execution process, and the duration of executing the cold start operation by the camera to be evaluated once is obtained in the above manner. Thus, N time periods for the camera to be evaluated to execute the cold start operation can be obtained, then the N time periods are averaged, and the average value is determined as the time period for the camera to be evaluated to execute the cold start operation.

In step S222, a fluency computation function corresponding to the scene type to which the test case belongs is acquired. The scene types comprise a starting type, a working mode switching type, a video recording type and a photographing type, and the fluency computing function corresponding to each scene type is obtained based on matlab simulation tool fitting.

The values illustrate that a technician can divide the operation of the user on the camera into four scene types, namely a start type, a switch type, a video recording type and a photographing type according to the historical data of the user on the camera to be evaluated. The operation mode switching type may include switching from a photographing mode to a video recording mode, switching from a video recording mode to a photographing mode, switching from a portrait mode to a default photographing mode, and so on.

In practical application, the requirements of users on the performance of cameras in different scenes are different, whether the smoothness of the operation of the camera in a certain scene is satisfactory or not can be obtained by acquiring feedback information of the use of the cameras by the users, and the smoothness of the operation of the camera in the certain scene is determined according to a preset rule. For example, when feedback information that the smoothness of the operation of the camera in a certain scene is unsatisfactory is received, the duration of the operation of the camera in the scene is obtained, for example, if the scene is a startup scene, the camera startup duration is obtained. And finally, fitting by a matlab simulation tool to obtain a corresponding formula of the starting duration and the score, and determining the formula as a fluency calculation function corresponding to the starting scene type. Thus, according to the above manner, the fluency computation function corresponding to each scene type can be obtained.

In a possible embodiment, the fluency computation functions corresponding to the start type scene and the switch type scene are the same, and the fluency computation function is shown in formula (1), where in formula (1), f1 (x _n ) Characterizing fluency corresponding to an nth test case of a camera to be evaluated, which belongs to a start type scene and a switching type scene, t1 _n And characterizing the duration of executing the operation corresponding to the nth test case belonging to the start type scene or the switching type scene.

The fluency computation function corresponding to the video type scene is shown as a formula (2), wherein f2 (x) in the formula (2) _n ) Characterizing fluency corresponding to an nth test case of a camera to be evaluated belonging to a video type scene, t2 _n And characterizing the duration of executing the operation corresponding to the nth test case belonging to the video type scene.

The fluency calculating function corresponding to the photographing type scene is shown as a formula (3), wherein f3 (x in the formula (3) _n ) Characterizing fluency corresponding to an nth test case of a camera to be evaluated, which belongs to a photographing type scene, t3 _n And characterizing the duration of executing the operation corresponding to the nth test case belonging to the photographing type scene.

In this way, the fluency computation function corresponding to each scene type can be predefined, and further, the fluency computation function corresponding to the scene type to which the test case belongs can be directly obtained when the performance of the camera is evaluated.

In step S223, the fluency of the camera to be evaluated corresponding to the test case is determined according to the fluency calculation function and the duration.

After the fluency computation function corresponding to the scene type to which the test case belongs is obtained, the duration obtained in the step S221 is input into the fluency computation function, so as to obtain the fluency of the camera to be evaluated corresponding to the test case.

By adopting the technical scheme, different fluency calculation functions are set for different scene types, so that the accuracy of determining the fluency of the camera to be evaluated, which corresponds to the test case, is further improved.

In practical applications, in order to further improve the comprehensiveness and accuracy of the camera performance evaluation, a plurality of test cases are generally used to evaluate the camera, so in one embodiment, the number of test cases is plural, as shown in fig. 4, and step S23 in fig. 2 may further include steps S231 to S233 to evaluate the performance of the camera to be evaluated according to the performance parameters of the camera to be evaluated corresponding to the test cases.

In step S231, for each test case, a value weight value of the test case is determined according to the frequency of use and/or the attention of the operation corresponding to the test case. The value weight value is used for representing the importance degree of the operation corresponding to the test case.

In the disclosure, according to feedback information of a user for each operation, the use frequency and/or attention degree of the user for each operation can be determined, and different values are set for test cases corresponding to operations with different use frequencies and/or attention degrees. For example, the value of the test case corresponding to the operation with the use frequency or the attention degree of more than 50% may be set to 10, the value of the test case corresponding to the operation with the use frequency or the attention degree of more than 5% and less than 50% may be set to 7, the value of the test case corresponding to the operation with the use frequency or the attention degree of more than 2% and less than 5% may be set to 4, the value of the test case corresponding to the operation with the use frequency or the attention degree of more than 1% and less than 2% may be set to 2, and the value of the test case corresponding to the operation with the use frequency or the attention degree of less than 1% may be set to 1. In this way, the value weight value of the test case corresponding to the operation with the use frequency or the attention degree of more than 50% is 10/24, the value weight value of the test case corresponding to the operation with the use frequency or the attention degree of more than 5% and less than 50% is 7/24, the value weight value of the test case corresponding to the operation with the use frequency or the attention degree of more than 2% and less than 5% is 4/24, the value weight of the test case corresponding to the operation with the use frequency or the attention degree of more than 1% and less than 2% is 2/24, and the value weight value of the test case corresponding to the operation with the use frequency or the attention degree of less than 1% is 1/24.

In step S232, the fluency of the camera to be evaluated is determined according to the value weight value of each test case and the fluency of the camera to be evaluated corresponding to each test case.

For example, assume that M (M is an integer greater than 3) test cases are used to evaluate the performance of the camera, where M1 (M1 is an integer greater than 1) test cases belong to a scene type that is a start type and a switch type, M2 (M2 is an integer greater than 1) test cases belong to a scene type that is a video type, and M3 (M3 is an integer greater than 1) test cases belong to a scene type that is a photo type, where m1+m2+m3=m. Then, aiming at m1 test cases of the start type scene and the switch type scene, fluency of the camera to be evaluated, which corresponds to the test cases, is respectively obtained to be f1 (x ₁ )、f1(x ₂ )、f1(x ₃ )……f1(x _m1 ) The corresponding value weight values are w respectively _1,1 、w _1,2 、w _1,3 ……w _1,m1 . Aiming at m2 test cases of video type scenes, fluency of the camera to be evaluated, which corresponds to the test cases, is respectively obtained to be f2 (x ₁ )、f2(x ₂ )、f2(x ₃ )……f2(x _m2 ) The corresponding value weight values are w respectively _2,1 、w _2,2 、w _2,3 ……w _2,m2 . Aiming at m3 test cases of a photographing type scene, fluency of the camera to be evaluated, which corresponds to the test cases, is respectively obtained to be f3 (x ₁ )、f3(x ₂ )、f3(x ₃ )……f3(x _m3 ) The corresponding value weight values are w respectively _3,1 、w _3,2 、w _3,3 ……w _3,m3 . The sum of the fluency corresponding to each test case and the product of the value weight values of the test cases can be determined as the fluency of the camera to be evaluated.

In step S233, the performance of the camera to be evaluated is evaluated according to the fluency of the camera to be evaluated.

In the present disclosure, a plurality of performance levels may be preset, and each performance level corresponds to a fluency interval, where a higher performance level indicates a faster speed of executing an operation by a camera, and a higher fluency of use by a user corresponds to a larger value of the fluency interval. Thus, after calculating the fluency of the camera to be evaluated in the above manner, the performance level of the camera to be evaluated may be determined according to the fluency and a smoothness interval corresponding to each performance level.

By adopting the technical scheme, the performance of the camera to be evaluated is evaluated by utilizing different test cases, the comprehensiveness of the camera performance evaluation is improved, and the accuracy of the camera performance evaluation is further improved.

The value is stated that in the disclosure, after the fluency of the camera to be evaluated is determined, the fluency can be output, so that a technician can know the advantages and disadvantages of the camera to be evaluated conveniently, and technical guidance is given to the technician in the development stage of the optimization of the camera performance.

In an exemplary development stage of the optimization of the camera performance, a technician can evaluate the performance of each version of the camera developed by the camera performance evaluation method and output the fluency of each version of the camera, so that the technician can monitor the state of each version conveniently, and further determine the camera version with the best performance.

Based on the same inventive concept, the present disclosure also provides a camera performance evaluation apparatus. Fig. 5 is a block diagram illustrating a camera performance evaluation apparatus according to an exemplary embodiment. As shown in fig. 5, the camera performance evaluation apparatus 500 may include:

a control module 501 configured to control a camera to be evaluated to execute a test case related to the camera to be evaluated in response to receiving an evaluation instruction for camera performance, the test case being used to describe an operation on the camera to be evaluated;

the acquisition module 502 is configured to monitor the execution process of the test case and acquire performance parameters of the camera to be evaluated, which correspond to the test case;

and the evaluation module 503 is configured to evaluate the performance of the camera to be evaluated according to the performance parameters of the camera to be evaluated, which correspond to the test case.

Optionally, the performance parameter is smoothness of the camera; the obtaining module 502 includes:

Optionally, the test cases are multiple; the evaluation module 503 includes:

Optionally, the control module 501 is configured to: in response to receiving an evaluation instruction for camera performance, controlling the camera to be evaluated to execute N times on the same test case, wherein N is an integer greater than or equal to 2;

Optionally, the obtaining module 502 includes:

Optionally, the third obtaining submodule includes:

The specific manner in which the various modules perform the operations in the apparatus of the above embodiments have been described in detail in connection with the embodiments of the method, and will not be described in detail herein.

The present disclosure also provides a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the steps of the camera performance assessment method provided by the present disclosure.

Fig. 6 is a block diagram illustrating a camera performance evaluation apparatus according to an exemplary embodiment. For example, apparatus 800 may be a mobile phone, computer, digital broadcast terminal, messaging device, game console, tablet device, medical device, exercise device, personal digital assistant, or the like.

Referring to fig. 6, apparatus 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input/output (I/O) interface 812, a sensor component 814, and a communication component 816.

The processing component 802 generally controls overall operation of the apparatus 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of a camera performance evaluation method. Further, the processing component 802 can include one or more modules that facilitate interactions between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to store various types of data to support operations at the apparatus 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phonebook data, messages, pictures, videos, and the like. The memory 804 may be implemented by any type or combination of volatile or nonvolatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.

The power component 806 provides power to the various components of the device 800. The power components 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the device 800.

The multimedia component 808 includes a screen between the device 800 and the user that provides an output interface. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may sense not only the boundary of a touch or slide action, but also the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and/or a rear camera. The front camera and/or the rear camera may receive external multimedia data when the apparatus 800 is in an operational mode, such as a photographing mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the device 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 further includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be a keyboard, click wheel, buttons, etc. These buttons may include, but are not limited to: homepage button, volume button, start button, and lock button.

The sensor assembly 814 includes one or more sensors for providing status assessment of various aspects of the apparatus 800. For example, the sensor assembly 814 may detect an on/off state of the device 800, a relative positioning of the components, such as a display and keypad of the device 800, the sensor assembly 814 may also detect a change in position of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, an orientation or acceleration/deceleration of the device 800, and a change in temperature of the device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscopic sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to facilitate communication between the apparatus 800 and other devices, either in a wired or wireless manner. The device 800 may access a wireless network based on a communication standard, such as WiFi,2G or 3G, or a combination thereof. In one exemplary embodiment, the communication component 816 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the apparatus 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic elements for performing the camera performance evaluation method.

In an exemplary embodiment, a non-transitory computer readable storage medium is also provided, such as memory 804 including instructions executable by processor 820 of apparatus 800 to perform a camera performance evaluation method. For example, the non-transitory computer readable storage medium may be ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

In another exemplary embodiment, a computer program product is also provided, comprising a computer program executable by a programmable apparatus, the computer program having code portions for performing the above-described camera performance evaluation method when executed by the programmable apparatus.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure. This application is intended to cover any adaptations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A camera performance evaluation method, comprising:

2. The method of claim 1, wherein the performance parameter is fluency of the camera;

the monitoring the execution process of the test case, obtaining the performance parameters of the camera to be evaluated, which correspond to the test case, includes:

3. The method of claim 2, wherein the test cases are a plurality of; the evaluating the performance of the camera to be evaluated according to the performance parameters of the camera to be evaluated corresponding to the test case includes:

4. The method of claim 2, wherein the controlling the camera under evaluation to execute the test case related to the camera under evaluation in response to receiving the evaluation instruction for the camera performance comprises:

5. The method according to claim 1, wherein the monitoring the execution process of the test case to obtain the performance parameter of the camera under evaluation corresponding to the test case includes:

6. The method of claim 5, wherein the acquiring the multi-frame image of the camera under evaluation executing the test case comprises:

7. A camera performance evaluation apparatus, comprising:

8. A camera performance evaluation apparatus, comprising:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to:

9. A computer readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the steps of the method of any of claims 1-6.

10. A camera performance evaluation system, comprising: user equipment, detection equipment, industrial cameras;

the detection device for performing the method of any one of claims 1-6.