WO2019119831A1

WO2019119831A1 - Method and apparatus for accessing camera, server, and readable storage medium

Info

Publication number: WO2019119831A1
Application number: PCT/CN2018/100543
Authority: WO
Inventors: 段庆领; 李军杰
Original assignee: 华为技术有限公司
Priority date: 2017-12-19
Filing date: 2018-08-15
Publication date: 2019-06-27
Also published as: CN108304247A

Abstract

A method and apparatus for accessing a camera, a server (101, 200), and a readable storage medium. A virtual machine with an Android operating system is deployed on the server (101, 200); the Android operating system provides a service terminal with a virtual desktop; a virtual camera is deployed at a hardware abstraction layer of the service terminal; the virtual camera is used for redirecting an access request as an access request of accessing a camera of a terminal device. The method for accessing a camera comprises detecting an access request of an application for accessing the virtual camera, redirecting the access request as the access request of accessing the camera of the terminal device, and then sending the redirected access request to a client of the terminal device. In this way, the client can access the camera of the terminal device according to the redirected access request. According to the method, a kernel layer of the service terminal does not need to participate, thereby saving the time taken, in the prior art, by accessing the camera of the terminal device by means of driving interaction with a kernel camera.

Description

Method and device for accessing camera, server, readable storage medium

Technical field

The present application relates to the field of computing, and in particular, to a method and apparatus for accessing a camera, a server, and a readable storage medium.

Background technique

Desktop virtualization (DV) technology virtualizes the desktop provided by the server, such as running a virtual machine (VM) or container that provides a virtual desktop on the server. The user can remotely access the virtual desktop provided by the server through the client on the terminal device, and the client can also redirect the operation of the user on the terminal device to the operation of the virtual desktop.

The mouse or keyboard on the terminal device can be redirected for virtual desktop access. Open source community targeting

The virtual machine of the operating system provides a virtual camera file "V4L2loopback"; the file can be deployed in the kernel of the virtual machine, so that the virtual machine can provide the V4L2 interface in the kernel state; the application on the virtual desktop can access the V4L2 The interface redirects access to the camera of the terminal device, for example to obtain image data from the camera.

Summary of the invention

In view of this, the present application provides a method and apparatus for accessing a camera, a server, and a readable storage medium, which facilitate a virtual machine of an Android operating system to access a camera of the terminal device.

In a first aspect, the present application provides a method of accessing a camera. The server deploys a virtual machine of the Android operating system, where the Android operating system provides a server of the virtual desktop, and a virtual camera is deployed at the hardware abstraction layer of the server, and the virtual camera is used to redirect the access request to the access terminal device. Camera access request.

In the method, the virtual camera detects an access request by the application to access the virtual camera, and redirects the access request to an access request of a camera accessing the terminal device. Then, the virtual camera sends the redirected access request to the client of the terminal device. In this way, the client can access the camera of the terminal device according to the access request obtained by the redirect.

All steps in the method do not require the kernel layer of the server to participate, thus avoiding the occupation of kernel resources and eliminating the time taken to access the camera of the terminal device by driving interaction with the camera of the kernel.

With reference to the first aspect or any of the possible designs of the first aspect, in a possible design, the virtual camera receives the parameters of the camera of the terminal device sent by the client, and the parameters of the camera of the terminal device include the The image format, resolution, and frame rate supported by the camera of the terminal device. The virtual camera configures the virtual camera using parameters of a camera of the terminal device.

In this way, the virtual camera is configured according to the parameters of the camera of the terminal device, and is convenient for redirecting the access request for accessing the virtual camera to the access request of the camera accessing the terminal device.

In combination with the first aspect or any of the possible designs of the first aspect, in a possible design, the virtual camera receives image data acquired by the client from the camera according to the redirected access request; the virtual camera The image data is sent to the application.

In this way, the application can acquire image data from the camera of the terminal device by accessing the request.

In combination with the first aspect or any of the possible designs of the first aspect, in the present possible design, the virtual camera establishes a communication connection between the virtual camera and the client.

Subsequently, the virtual camera and the client can transmit data through the communication connection.

In a second aspect, the present application provides an apparatus for accessing a camera, the apparatus comprising functional units implementing the method of accessing a camera provided by the first aspect or various possible designs of the first aspect.

In a third aspect, the application provides a server, where the server includes a processor and a memory;

The processor executes instructions stored in the memory such that the server performs the method of accessing a camera provided by the first aspect or various possible designs of the first aspect.

In a fourth aspect, the application provides a computer readable storage medium. Instructions are stored in the computer readable storage medium. The server executes instructions in the computer readable storage medium such that the server performs the method of accessing the camera provided by the first aspect or various possible designs of the first aspect.

In a fifth aspect, the application provides a computer program product comprising instructions stored in a computer readable storage medium. The server executes instructions in the computer program product such that the server performs the method of accessing the camera provided by the first aspect or various possible designs of the first aspect.

DRAWINGS

FIG. 1 is a schematic diagram of an application scenario including a server and a terminal device;

2 is a schematic diagram of a hardware architecture of a server;

3 is a schematic diagram of a logical architecture of a server and a client;

4 is a schematic flowchart of accessing a camera according to an embodiment of the present invention;

FIG. 5 is a schematic structural diagram of an apparatus for community discovery according to an embodiment of the present invention.

Detailed ways

The technical solutions provided by the embodiments of the present invention are described below in conjunction with the accompanying drawings in the embodiments of the present invention.

Application scenario

Referring to FIG. 1, the server 101 is communicatively coupled to the terminal device 102. The terminal device 102 can be a thin client (TC), a mobile terminal, a tablet or other device.

The server 101 deploys a virtual machine (VM) of an Android (Android) operating system. The VM provides a virtual desktop to the terminal device 102. The terminal device 102 deploys a client of the virtual desktop through which the virtual desktop provided by the VM to the terminal device 102 is accessed.

Optionally, the server 101 is a server that provides virtual desktops in the cloud, and FIG. 1 only schematically shows one server 101 in the cloud.

It can be understood that the cloud can also include multiple servers or even a large number of servers. Servers in the cloud are virtualized by cloud computing, providing virtualized computing resources, storage resources, input/output (I/O) resources, and more.

A server in the cloud that can deploy a VM running an application and a VM that provides a virtual desktop. It can be understood that the VM that runs the running application and the VM that provides the virtual desktop can be located on the same server 101; or the VM that provides the virtual desktop is located at the server 101, and the VM that runs the application is located at the other server. The virtual desktop provides the application to the client, and the application accesses the camera of the terminal device through the virtual desktop.

FIG. 2 schematically provides one possible basic hardware architecture of server 101.

Referring to FIG. 2, the server 200 includes at least one processor 201, a memory 202, a communication interface 203, and a bus 204.

Each processor 201 in the server 200 may be any one of the following: a central processing unit (CPU), an ARM processor, a Field Programmable Gate Array (FPGA), a dedicated processor. Such devices with computational processing capabilities.

Alternatively, the number of processors 201 in the server 200 is plural; the plurality of processors 201 may be different types of processors 201, or may be the same type of processor 201. In an optional implementation manner, the plurality of processors 201 may also be integrated into a many-core processor.

Memory 202 stores computer instructions and data; for example, memory 202 stores computer instructions and data associated with an operating system (OS). The memory 202 may be any one or any combination of the following: a read only memory (ROM), a non-volatile memory (NVM), a solid state drive (SSD), Storage media such as mechanical hard disks, disks, and disk arrays.

The communication interface 203 may be any one or any combination of the following: a network interface (such as an Ethernet interface), a wireless network card, or the like having a network access function.

The communication interface 203 is used by the server 200 to perform data interaction with other devices (e.g., terminal devices).

The bus 204 includes a memory bus, and may also include any one or any combination of the following: an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, and an extended industry. A device for wired data transmission such as an Extended Industry Standard Architecture (EISA) bus. Figure 2 shows the bus 204 with a thick line.

The processor 201, the memory 202, and the communication interface 203 are connected by a bus 204. Thus, processor 201 can access memory 202 via bus 204 and interact with other devices (e.g., terminal devices) via communication interface 203 via bus 204.

In the embodiment of the present invention, the server 200 executes the instructions in the memory 202, so that the server 200 implements the method for accessing the camera provided by the method embodiment.

Redirecting embodiment

The VM running on the server 101 deploys an Android (Android) operating system. The VM provides a virtual desktop for the Android operating system.

Referring to FIG. 3, the Android operating system deploys a server of a virtual desktop, and the server includes a kernel layer, a hardware abstraction layer (HAL), and a framework layer. The server can also include an application layer.

The kernel layer provides a camera driver that the application can call to access the local camera or the camera of the terminal device. However, accessing the camera of the terminal device through the driver of the kernel layer requires interaction with the kernel, which occupies kernel resources, and the interaction time brings delay in accessing the camera. Therefore, the embodiment of the present invention provides a virtual camera in a hardware abstraction layer (HAL), which replaces the driver of the kernel layer to implement access to the camera of the terminal device, thereby avoiding occupation of kernel resources, and eliminating the need for The time taken by the driver's interaction of the kernel's camera.

Referring to Figure 3, the hardware abstraction layer provides a virtual camera. The virtual camera is used to redirect an application's access request to an access request to access the camera of the terminal device. Specifically, the virtual camera includes a communication module and a management module.

The communication module (ie, the communication module of the server) is configured to establish a communication connection with the client, and specifically may establish a communication connection with the communication module of the client. Subsequently, the communication module can interact with the communication module of the client, such as a communication module such as a client to send an access request.

The management module (the management module of the server) is used to manage the virtual camera. When the communication module of the server establishes a communication connection with the client, the management module authenticates the client and authenticates, and then allows a communication connection with the communication module of the client. The way to authenticate can be the way to use tokens for authentication.

The camera layer provides a camera service module. When the application needs to access the virtual camera, it first establishes communication with the camera service module, and then opens the virtual camera through the camera service module, thereby implementing the application's access to the virtual camera.

The application layer can deploy applications. Of course, the application can also be deployed in other VMs or containers or physical servers or other environments where applications can be deployed. The application here can access the virtual camera through the camera service module.

Referring to FIG. 3, the terminal device deploys a client of the virtual desktop. The client includes a communication module, a management module, a detection module, and a message processing module.

The communication module of the client is used to request to establish a communication connection with the communication module of the server. Thereby, the communication connection between the client and the server is realized.

The management module of the client is used to manage the camera of the terminal device. The management module can acquire the parameters of the camera, and can send the parameters of the camera to the server through the communication module of the client. The management module can control the camera, for example, can control the opening or closing of the camera.

The detection module of the client is used to identify the plugging and unplugging state of the camera, specifically whether the camera is normally inserted or unplugged. When detecting that the camera is normally inserted, the communication module notifies the server to load the virtual camera mapped by the camera on the server; when detecting that the camera is unplugged, the communication module notifies the server to uninstall the camera at the server. Map the virtual camera.

The message processing module of the client is configured to identify the request of the server, and instruct the management module of the client to perform a corresponding response operation according to the request of the server. For example, according to the access request of the server, the management module of the client is instructed to acquire image data from the camera; and the image data is fed back to the server through the communication module.

Method embodiment

Referring to FIG. 4, a method for accessing a camera provided by an embodiment of the present invention includes step S401, step S402, step S403, step S404, step S405, step S406, step S407, and step S408.

A virtual machine camera is deployed in the hardware abstraction layer of the server.

In step S401, the virtual camera establishes a communication connection with the client.

Specifically, the communication module of the client requests to establish a communication connection with the communication module in the virtual camera, and if the virtual camera authenticates the client, the communication module of the client continues to communicate with the communication module in the virtual camera. The establishment of the connection.

The communication connection is for transmitting data between the virtual camera and the client.

It should be understood that, in implementing the method of accessing the camera, if the establishment of the communication connection between the virtual camera and the client has been previously completed, step S401 may be omitted. That is, step S401 is an optional step in the method of accessing the camera.

Step S402, the client sends the parameters of the camera of the terminal device to the virtual camera.

The camera of the terminal device may be a camera deployed on the terminal device, or may be a camera connected to the terminal device, and the camera may be accessed by the terminal device.

The parameters of the camera include an image format, resolution, and frame rate supported by the camera.

The virtual camera receives the parameters of the camera of the terminal device sent by the client, and stores the parameters of the camera of the terminal device.

For example, the management module of the client acquires the parameters of the camera of the terminal device, and the client sends the parameter to the communication module in the virtual camera through its communication module. The communication module in the virtual camera receives the parameter, and the management module in the virtual camera stores the parameter.

In step S403, the virtual camera self-configures using the parameters of the camera of the terminal device.

Specifically, the management module in the virtual camera completes parameter configuration of the virtual camera according to parameters of the camera of the terminal device.

It should be noted that, when implementing the method of accessing the camera, if the parameter configuration of the virtual camera has been previously completed, step S402 and step S403 may be omitted. That is, step S402 and step S403 are optional steps in the method of accessing the camera.

Step S404, the virtual camera detects an access request of the application to access the virtual camera.

Applications can run on servers or other servers that provide virtual desktops. Specifically, the application can run on the operating system of the server, can also run in the VM or container deployed by the server, and can also run in other environments in the server, where the scenario for running the application is not performed. limited.

The app can access the virtual camera in the HAL. Specifically, the application sends an access request to the virtual camera by providing a camera service module provided by the server of the virtual desktop. Thus, the virtual camera can detect an access request for the application.

Optionally, before the application accesses the virtual camera, the virtual camera provides the virtual camera with parameters of the virtual camera (ie, parameters of the camera of the terminal device), and the application can access the data according to the parameter. Go to the virtual camera.

In step S405, the virtual camera redirects access to the virtual camera access request.

Specifically, the management module in the virtual camera redirects the access request to access the virtual camera to an access request of the camera accessing the terminal device.

Optionally, the technology for redirecting the access request of the virtual camera may be similarly referred to the prior art technology for redirecting access requests of other devices, for example, referring to the access request of the mouse or keyboard of the redirected access terminal device. Related technology.

Step S406, the virtual camera sends the access request obtained by the redirect to the client.

Specifically, the communication module in the virtual camera transmits the access request to the communication module of the client through a communication connection between the client and the virtual camera.

The client receives the access request through its communication module.

In step S407, the client acquires image data from the camera of the terminal device.

Specifically, the message processing module in the client recognizes the access request. Since the access request indicates that the image data of the camera is acquired, the message processing module instructs the management module to acquire image data from the camera of the terminal.

S408. The client sends the image data to the virtual camera.

Specifically, the client transmits the image data to the communication module of the virtual camera through its communication module.

Subsequently, the virtual camera can feed the image data back to the application through the camera service module. Thus, by performing steps S404, S405, S406, S407, and S408, the application can acquire image data from the camera of the terminal device by the access request.

It should be noted that, when implementing the method of accessing the camera, if it is not necessary to feed back the image data to the virtual camera according to the access request, step S407 and step S408 may be omitted. That is, step S407 and step S408 are optional steps in the method of accessing the camera.

In addition, if the camera of the terminal device has a parameter update, the client will feed back the updated parameters to the virtual camera. The virtual camera updates the configuration of the virtual camera based on the parameter.

From the point of view of accessing the camera, all steps do not require the kernel layer of the server to participate, thus avoiding the occupation of kernel resources and eliminating the time taken to access the camera of the terminal device by driving interaction with the camera of the kernel.

Apparatus and apparatus embodiment

An embodiment of the present invention provides a device for accessing a camera. Referring to FIG. 5, the device 500 includes functional units for implementing the steps included in the method for accessing a camera provided by the method embodiment. The embodiment of the present invention is in the device 500. How to divide functional units is not limited, and the following is exemplarily provided for division of functional units, as shown in FIG.

As shown in FIG. 5, the apparatus 500 for accessing a camera is deployed on a server end of the Android operating system to provide a virtual desktop, and the hardware abstraction layer of the server is used to redirect the access request to the camera of the access terminal device. Access request; the apparatus 500 includes:

The management module 501 is configured to detect an access request of the application to access the virtual camera, and redirect the access request to an access request of a camera accessing the terminal device;

The communication module 502 is configured to send the redirected access request to the client of the terminal device, where the client is configured to access the camera of the terminal device according to the access request obtained by the redirecting.

Optionally, the communication module 502 is configured to receive a parameter of a camera of the terminal device that is sent by the client, where a parameter of the camera of the terminal device includes an image format and a resolution supported by a camera of the terminal device. And frame rate;

The management module 501 is configured to configure the virtual camera by using parameters of a camera of the terminal device.

Optionally, the communication module 502 is configured to:

Receiving image data acquired by the client from the camera according to the access request obtained by the redirecting;

The image data is sent to the application.

Optionally, the communication module 502 is configured to establish a communication connection between the virtual camera and the client, where the communication connection is used to transfer data between the virtual camera and the client.

In the embodiment of the present invention, the function description of each functional unit included in the device 500, refer to the foregoing embodiments (for example, the application scenario embodiment, the redirection embodiment, and the method embodiment), and the related steps included in the method for accessing the camera. .

Optionally, the device 500 for accessing the camera may be a virtual camera as described in the embodiment of the present invention. The virtual camera is deployed in the hardware abstraction layer described in the embodiment of the present invention.

Referring to FIG. 2, an embodiment of the present invention provides a server 200. The server includes a processor 201 and a memory 202. The processor 201 executes instructions stored in the memory 202, so that the server 200 executes the method for accessing the camera provided by the method embodiment.

An embodiment of the present invention provides a readable storage medium (such as the memory 202 in FIG. 2), where the readable storage medium stores instructions, and the server executes the instructions, so that the server executes the access camera provided by the method embodiment. Methods.

Embodiments of the present invention provide a computer program product comprising instructions stored in a computer readable storage medium. The server executes instructions in the computer program product such that the server performs the method of accessing the camera provided by the method embodiment.

It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not limited thereto; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that The technical solutions described in the foregoing embodiments are modified, or some of the technical features are equivalently replaced; and the modifications or substitutions do not deviate from the technical scope of the embodiments of the present invention.

Claims

A method for accessing a camera, wherein the virtual machine deploys an Android operating system, the Android operating system provides a server of the virtual desktop, and a virtual camera is deployed in the hardware abstraction layer HAL of the server, and the virtual camera is used. Redirecting the access request to an access request of a camera accessing the terminal device; the method includes:

Detecting an application access request by the application to access the virtual camera;

Redirecting the access request to an access request of a camera accessing the terminal device;

Sending the redirected access request to the client of the terminal device, the client is configured to access the camera of the terminal device according to the access request obtained by the redirecting.
The method of claim 1 wherein the method comprises:

Receiving, by the client, a parameter of a camera of the terminal device, where a parameter of a camera of the terminal device includes an image format, a resolution, and a frame rate supported by a camera of the terminal device;

The virtual camera is configured using parameters of a camera of the terminal device.
The method of claim 1 or 2, wherein the method comprises:

Receiving image data acquired by the client from the camera according to the access request obtained by the redirecting;

The image data is sent to the application.
The method according to any one of claims 1 to 3, wherein the method comprises:

Establishing a communication connection between the virtual camera and the client, the communication connection for transmitting data between the virtual camera and the client.
An apparatus for accessing a camera, wherein the virtual machine deploys an Android operating system, the Android operating system provides a server of the virtual desktop, and a virtual camera is deployed in the hardware abstraction layer HAL of the server, and the virtual camera is used by the virtual camera. Redirecting the access request to an access request of a camera accessing the terminal device; the device includes:

a management module, configured to detect an access request of the application to access the virtual camera, and redirect the access request to an access request of a camera accessing the terminal device;

And a communication module, configured to send the redirected access request to a client of the terminal device, where the client is configured to access a camera of the terminal device according to the access request obtained by the redirecting.
The device according to claim 5, characterized in that

The communication module is configured to receive a parameter of a camera of the terminal device that is sent by the client, where a parameter of the camera of the terminal device includes an image format, a resolution, and a frame rate supported by a camera of the terminal device;

The management module is configured to configure the virtual camera by using parameters of a camera of the terminal device.
The device according to claim 5 or 6, wherein the communication module is configured to:

Receiving image data acquired by the client from the camera according to the access request obtained by the redirecting;

The image data is sent to the application.
A device according to any one of claims 5 to 7, wherein

The communication module is configured to establish a communication connection between the virtual camera and the client, and the communication connection is used to transfer data between the virtual camera and the client.
A server, characterized in that the server comprises a processor and a memory;

The processor executes the instructions stored in the memory such that the server performs the method of accessing a camera as claimed in any one of claims 1 to 4.
A readable storage medium, wherein the readable storage medium stores instructions, and the server executes the instructions to cause the server to perform the method of accessing a camera according to any one of claims 1 to 4.