CN117453461A - Method for processing image exception of intelligent glasses optical machine, computing device and storage medium - Google Patents

Abstract

The present disclosure relates to a method, a computing device, a storage medium, and the like for handling an abnormal image of an optical machine of an intelligent glasses. The method comprises the following steps: reading a register representing parameters of an output picture of the optical machine; determining that the optical machine has picture abnormality according to the value of the register; and executing a reset operation on the optical machine. By utilizing the method disclosed by the invention, the problem of abnormal optical machine pictures can be accurately and automatically solved.

Description

Method for processing image exception of intelligent glasses optical machine, computing device and storage medium

Technical Field

The present disclosure relates to methods, computing devices, storage media, and the like for handling smart eyewear opto-mechanical frame anomalies.

Background

At present, an optical machine picture is abnormal in some cases in the intelligent glasses product, especially when the electrostatic test of the whole machine is performed, the picture is abnormal and can not be automatically recovered, so that the EMC (Electro Magnetic Compatibility ) authentication test (such as no artificial participation in automatic recovery within 10 seconds) is not satisfied.

Disclosure of Invention

The disclosed embodiments provide methods of handling smart eyewear opto-mechanical frame anomalies and corresponding computing devices and non-transitory machine-readable storage media performing these methods.

According to a first aspect of an embodiment of the present disclosure, there is provided a method for processing an abnormal image of an optical machine of a smart glasses, including: reading a register representing a parameter of an output picture of the optical machine; determining that the optical machine has picture abnormality according to the value of the register; and executing a reset operation on the optical machine.

Optionally, the registers include a first register, a second register, and/or a third register, wherein: the first register records a check value of the data of the output picture, the second register records a frame counter value of the output picture or data representing a frame rate of the output picture, and the third register records a resolution value of the output picture.

Optionally, the value recorded by the first register is read, and if it is determined that the data of the output picture is inconsistent with the data of the picture input to the optical machine according to the check value of the data of the output picture, it is determined that the optical machine has a picture abnormality.

Optionally, the value recorded by the second register is read, and in the case that the frame counter value or the data representing the frame rate is inconsistent with the corresponding frame count value or data obtained according to the frame rate of the picture input to the optical engine, it is determined that the optical engine has a picture abnormality.

Optionally, reading a value recorded by the third register, and determining that the optical engine has a frame abnormality if the resolution value is inconsistent with a resolution value of a frame input to the optical engine.

Optionally, the reading sequence of the first register, the second register and the third register is set, and in the case that it is determined that the optical engine has a frame abnormality according to a value of any one register, the subsequent reading of the registers is stopped, and a reset operation is performed on the optical engine.

Optionally, a time period for timing reading the register is set according to the electrostatic detection requirement.

Optionally, performing a reset operation on the optical bench includes: and executing power-on and power-off reset operation on the optical machine.

According to a second aspect of embodiments of the present disclosure, there is provided a computing device comprising: a processor; and a memory having executable code stored thereon which, when executed by the processor, causes the processor to perform the method according to any of the aspects described above.

According to a third aspect of embodiments of the present disclosure, there is provided a non-transitory machine-readable storage medium having stored thereon executable code, which when executed by a processor of an electronic device, causes the processor to perform the method as set forth in any one of the aspects described above.

Drawings

The foregoing and other objects, features and advantages of the disclosure will be apparent from the following more particular descriptions of exemplary embodiments of the disclosure as illustrated in the accompanying drawings wherein like reference numbers generally represent like parts throughout exemplary embodiments of the disclosure.

Fig. 1 schematically illustrates a structural diagram of smart glasses in accordance with at least one embodiment of the present disclosure.

Fig. 2 schematically illustrates a flow diagram of a method of handling smart eyeglass opto-mechanical frame anomalies in accordance with at least one embodiment of the present disclosure.

Fig. 3 schematically illustrates a flow diagram of a method of handling smart eyeglass opto-mechanical frame anomalies in accordance with at least one embodiment of the present disclosure.

Fig. 4 schematically illustrates a structural diagram of a computing device in accordance with at least one embodiment of the present disclosure.

Detailed Description

Preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While the preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

It should be appreciated by those skilled in the art that the terms "first," "second," and the like in the description, claims, and drawings of this disclosure are used for distinguishing between similar objects and not for describing a particular sequence or order, and that no additional limitation is present.

Smart glasses are one type of wearable smart product that may include XR (Extended Reality) glasses, etc. XR glasses may in turn include AR (Augmented Reality) glasses, VR (Virtual Reality) glasses, MR (Mixed Reality) glasses, and the like.

As shown in fig. 1, in some examples, the smart glasses may embed hardware modules in their frames (including the temples), including a processing system and an optical engine, where the processing system may control the optical engine and transmit information about a frame to be viewed by a user to the optical engine, and the optical engine may convert the received frame information into an optical signal, form an output frame, and project it onto a lens or into an eye of the user, for example, through optical elements. Those skilled in the art will appreciate that fig. 1 is merely exemplary and not limiting of the structure of the smart glasses of the present disclosure; for example, a processing system may include a processor, an internal memory, and a serial bus connecting the processor and the internal memory, where the processor may include one or more processing units, such as an application processor (application processor, AP), a modem processor, a graphics processor (graphics processingunit, GPU), an image signal processor (image signal processor, ISP), a controller, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, etc., and where the internal memory may be used to store computer executable program code including instructions. The internal memory may include a stored program area and a stored data area. The storage program area may store an application program (such as a sound playing function, an image display function, etc.) required for at least one function of the operating system, and the like. The storage data area may store data (e.g., audio data, phonebook, etc.) created during use of the smart glasses.

In one example, the processing system may be a micro-control unit that integrates a processor, internal memory, external memory, and I/O interfaces, with bus connections used between the processor, internal memory, external memory, and I/O interfaces.

In one example, the processing system and hardware such as the opto-mechanical may also be attached to the glasses rather than embedded therein, and so on.

However, as described above, in some cases, particularly when the static test is performed, the smart glasses have a problem that the output image of the optical machine is abnormal and cannot be automatically recovered. At present, a stable and effective scheme for electrostatic abnormality of intelligent glasses products is not found, but the traditional scheme for electrostatic abnormality is difficult to be applied to intelligent glasses. For example, conventional solutions mainly include metal structure electrical connection design and electrostatic discharge circuit design inside the product, however: auxiliary materials such as copper foil can be used in structural electric connection design, the internal structural space of the intelligent glasses is difficult to meet, and the requirement of light weight of intelligent glasses products is not met; the static discharge circuit design requires a large block of discharge metal inside the product, which is difficult to realize in smart glasses. In summary, the electrostatic problem is a major challenge for smart eyewear products.

In view of this, the present disclosure proposes a solution for handling abnormal images of an optical engine of an intelligent glasses, which can have the advantages of satisfying the light weight and low cost of the product. In some examples, the method of the present disclosure may be implemented in a software manner by a computer program, which is particularly advantageous for the photo frame abnormality and the corresponding recovery caused by static electricity, and can meet the requirement of static electricity test.

An exemplary scheme for handling the picture abnormality of the smart eyeglass ray machine will be described in detail with reference to the accompanying drawings, so that the concept of the present disclosure will be clearly demonstrated. It should be understood that the present disclosure is not limited to these exemplary schemes.

Fig. 2 schematically illustrates a flow diagram of a method of handling smart eyeglass opto-mechanical frame anomalies in accordance with at least one embodiment of the present disclosure.

As shown in fig. 2, at step S210, a register representing a parameter of an output screen of the optical bench is read. Then, at step S220, it is determined that the light engine has a frame abnormality according to the value of the register.

The present disclosure finds that, in some embodiments, the abnormal image of the optical engine can be mainly divided into three cases: the abnormal conditions can be judged by the values of one or more registers originally arranged in the optical machine, the registers respectively record related parameters of an output picture of the optical machine, and whether the output picture of the optical machine is abnormal or not can be judged by analyzing the parameters (such as comparing with the parameters of an input picture) so as to read the values of corresponding registers in the optical machine by using a processing system for example.

The following three cases of screen anomalies are discussed in more detail respectively (it should be understood that the following specific operational details should not be considered as limiting the present disclosure):

1. freezing screen:

when the screen freezing condition occurs, the picture input to the optical machine by the main board end (such as the processing system) is inconsistent with the picture output by the optical machine. This makes it possible to determine the abnormality by comparing the data of the output screen and the input screen of the camera. The data of the screen here may include a value of each pixel in the screen, a control value associated with each pixel, or the like. The optical engine can continuously receive and buffer the data of each frame of picture input by the processing system, and then sequentially output the data of each frame of picture to the electro-optical conversion device so as to output corresponding pictures, and when abnormal conditions occur, the output data can change relative to the input data, so that the output picture is inconsistent with the input picture. Therefore, whether or not a change occurs can be judged by checking the data of the output screen.

For example, the processing system further includes a CRC (Cyclic Redundancy Check ) check register, which records a check value obtained by performing CRC check on data of an output screen of the optical engine.

In a CRC check method, the whole data (including the check code portion) may be subjected to a CRC check operation to obtain a remainder, and the result is stored in a CRC check register, and if the data does not change, the obtained check value (i.e., the remainder) should be 0, i.e., the value of the register should be "00000000"; in another CRC check method, only the actual data (excluding the check code portion) may be subjected to CRC calculation, the result may be stored in a CRC check register, and compared with the CRC check code of the input data to determine the consistency. When the former CRC check method is adopted, the register value '00000000' of the CRC check register can be stored in a database of a main board when the optical machine normally works, when the optical machine output picture is checked every time later, a check result is written into the register, the processing system reads the check value written in the register and compares the check value (000000000000) with the stored check value, for example, an addition operation or a subtraction operation is carried out, and when the operation result is not '00000000', the picture output by the optical machine is abnormal. It will be appreciated that the above-described stored check values are stored in the database of the motherboard itself, and the loss of the stored values due to the update or uninstallation of the system or the change in external storage space can be prevented.

Alternatively, in the latter checking mode, the processing system may also write the check value obtained by performing CRC check on the data of the picture output by the main board end to the optical engine into the register, that is, the check value written into the register each time may be different, there may be two different eight-bit registers, for example, the eight-bit register a and the eight-bit register B, the processing system writes the check value obtained by performing CRC check on the data of the picture output by the main board end to the optical engine into the register a, and the processing system writes the check value obtained by performing CRC check on the data of the picture output by the optical engine into the register B, and when comparing, the processing system reads the data of the register a and the data of the register B to perform subtraction operation, and when the operation result is not 0, it indicates that the picture output by the optical engine is abnormal. In another example, there may be only one 16-bit register C, where the CRC check value is written into the upper eight bits and the lower eight bits of the register C twice, respectively, and the processing system reads the upper eight bits and the lower eight bits of the register C to perform subtraction operation when the comparison result is not 0, and indicates that the picture output by the optical machine is abnormal.

Although the preceding describes the marking of a freeze event by a CRC check register for picture comparison, it should be understood that the register is not limited thereto, as may a register that also includes other types of check values that record data of the output picture.

2. And (3) clamping:

when a clamping event occurs, the frame rate of the input picture at the main board end is inconsistent with the frame rate of the picture output by the optical machine. This makes it possible to determine the abnormality by comparing the frame rates of the output screen and the input screen of the camera. A stuck event may be marked, for example, by a register that records the frame counter value.

For example, the processing system has a frame counter that counts the output frames of the light engine and a register that stores the value of the frame counter. The processing system can read the register at regular time to obtain the frame counter values of the front and the back two times, estimate the difference value between the frame counter values of the front and the back two times under normal conditions according to the interval time of the front and the back two times of reading and the frame rate of the picture sent to the optical machine by the processing system, then compare the difference value between the actually obtained frame counter values with the estimated difference value, and can confirm that the frame rates of the two are inconsistent under inconsistent conditions and that the picture of the optical machine is abnormal. After each reading comparison, the frame counter value read at this time can be covered with the frame counter value read at last time stored in the processing system to serve as a basis of next comparison, namely, compared with the frame counter value read at next time, the frame counter value read at this time becomes the frame counter value of the previous time. Or the main board can also count the same frames of the frames output to the optical machine, then compare the read frame counter value of the optical machine with the frame count value of the main board, and confirm that the optical machine frame is abnormal under the condition of inconsistent.

3. Black screen:

when a black screen event occurs, the opto-mechanical resolution register is cleared. The resolution register records the resolution value of the current output picture when the optical machine works normally, for example, the value representing 480P/720P/1080P and the like; in contrast, in the case of a black screen, the output screen does not output pixels, and the output becomes 0, so that the value of the resolution register also becomes 0. The determination of the abnormal situation can thus be made by comparing the values of the resolution registers, i.e. a black screen event can be marked by comparing the values of the resolution registers.

For example, the value of the resolution register (for example, "000X") when the optical engine works normally may be stored in the database of the motherboard, the value read from the register by the processing system each time later is compared with the stored value, and if the values are inconsistent, it is determined that the optical engine picture is abnormal; alternatively, instead of storing the normal operating value, the processing system may directly detect whether the register value read each time is zero, and if a zero value is detected, this indicates that there is an opto-mechanical frame abnormality.

In summary, the abnormal condition of the optical engine can be determined by reading and analyzing one or more of the 3 registers as required.

For example, in some embodiments, the registers in the foregoing steps of fig. 2 may include at least one of a first register, a second register, and a third register, wherein: the first register records a check value of data of the output picture, the second register records a frame counter value of the output picture or data representing a frame rate of the output picture, and the third register records a resolution value of the output picture. In some examples, the operations of reading and determining whether the light engine frame is abnormal may be performed on a plurality or all of the above-mentioned registers simultaneously or in a preset order. When it is determined that the optical engine has a frame abnormality according to the value of any one of the registers, the reading and judging operations of the other registers can be stopped without performing the related operations of the other registers. In addition, in some examples, the time period for timing the reading of each register may be set as desired (e.g., based on electrostatic detection requirements).

In the case of reading the first register, in some examples (for example, in the first CRC check mode described above), the processing system may determine whether the data of the output frame is identical to the data of the frame input to the optical engine directly according to the value of the first register, while in other examples, the processing system may compare the check value recorded by the first register with the check value of the data of the frame input to the optical engine, and in the case of inconsistency, determine that the optical engine has a frame abnormality (possibly a frozen screen condition).

In the case where the second register records the frame counter value of the output picture, as described in the previous example, in the case where the recorded frame counter value is inconsistent with the corresponding frame count value obtained from the frame rate of the picture input to the optical engine (i.e., the frame count value at the time of normal operation), it is determined that the optical engine has a picture abnormality; in the case where the second register records data representing the frame rate of the output picture, which may be any data obtained by the optical engine that can represent the frame rate of its output picture, for example, the optical engine may obtain the number of frames output in one second using the frame counter and store it as data representing the frame rate in the register, and in the case where the data representing the frame rate of the output picture does not coincide with the corresponding data representing the frame rate of the picture input to the optical engine, it is determined that the optical engine has a picture abnormality.

In the case of reading the third register, as described above, the processing system may determine that the resolution of the output frame is inconsistent directly from the value of the third register in some examples, thereby determining that the frame is abnormal in the light engine, and in other examples, the processing system may compare the resolution value recorded by the third register with the resolution value of the frame input to the light engine, and in the event of inconsistency, determine that the frame is abnormal in the light engine.

Then, as shown in fig. 2, at step S230, in the case where it has been determined in the foregoing step that there is a screen abnormality in the optical pickup, a reset operation is performed on the optical pickup.

In some embodiments, a power-on-power-off reset operation may be performed on the optical machine. For example, the Power-On Reset operation may include two steps, i.e., power-On Reset (POR) and Power-off Reset, where the Power-On Reset can cause the light engine to jump out of the current abnormal operating state and initialize back to the normal operating state.

It should be understood that the resetting operation is not limited thereto, and for example, a dedicated resetting program may be set for the aforementioned different types of screen abnormalities within the processing system or the optical engine, as long as the output screen of the optical engine after resetting can be made to coincide with the input screen.

In addition, in some embodiments, in order to meet EMC authentication criteria requirements (such as a requirement of no human participation in automatic recovery within 10 seconds) and reduce power consumption at the same time, the operations of reading the three registers and judging whether the picture of the optical machine is abnormal may be performed in turn at regular time, and if any one step determines that the picture of the optical machine is abnormal, the subsequent steps are stopped, and the reset operation is performed on the optical machine. Fig. 3 shows a schematic flow chart of an example of such a method.

Since the operations of reading and judging whether or not the light engine screen is abnormal can be sequentially performed on the first to third registers in a predetermined arbitrary order, any one of the first to third registers is denoted by a register A, B, C in fig. 3, respectively.

As shown in fig. 3, in step S310, the register a is read, then in step S320, it is determined accordingly whether there is an abnormality in the optomechanical output screen according to the value of the register a, if so, the process proceeds to step S370, that is, the optomechanical reset operation is performed accordingly, and if not, the process ends, and if so, the subsequent register reading is performed subsequently, that is, the process proceeds to step S330.

In step S330, the register B is read, then in step S340, it is determined whether there is an abnormality in the optical engine output screen according to the value of the register B, and if so, the process proceeds to step S370, i.e., a corresponding reset operation is performed on the optical engine, and if not, the process ends, and then the subsequent register reading is performed, i.e., the process proceeds to step S350.

In step S350, the register C is read, then in step S360, it is correspondingly determined whether there is an abnormal condition of the optical machine output screen according to the value of the register C, and if so, the process proceeds to step S370, that is, the optical machine is subjected to a corresponding reset operation, and the flow ends, and if not, the process waits for a preset time interval to perform the next cycle.

The details of the operations of the foregoing steps may be adopted by the examples or embodiments described in connection with fig. 2, and are not described herein.

The time period of the timing cycle in fig. 3 may be set according to the electrostatic detection requirement. For example, the period may be set to 5 seconds, and the time required for the reset operation of the optical machine is generally about 1 second, so that the normal state can be recovered within 10 seconds, which can satisfy the EMC certification requirements.

The method meets EMC authentication criteria, and solves the problem that the static event in the northern dry area frequently causes abnormal picture using experience.

Fig. 4 illustrates a schematic structural diagram of a computing device that may be used to implement the method of handling smart eyeglass opto-mechanical frame abnormalities described above, in accordance with at least one embodiment of the present disclosure.

Referring to fig. 4, a computing device 400 includes a memory 410 and a processor 420.

For example, the computing device may be a smart glasses, a processing system therein, or a module containing a processing system, etc., as described above.

Processor 420 may be a multi-core processor or may include multiple processors. In some embodiments, processor 420 may comprise a general-purpose host processor and one or more special coprocessors, such as a Graphics Processor (GPU), digital Signal Processor (DSP), or the like. In some embodiments, the processor 420 may be implemented using custom circuitry, for example, an application specific integrated circuit (ASIC, application Specific Integrated Circuit) or a field programmable gate array (FPGA, field Programmable Gate Arrays).

Memory 410 may include various types of storage units, such as system memory, read Only Memory (ROM), and persistent storage. Where the ROM may store static data or instructions that are required by the processor 420 or other modules of the computer. The persistent storage may be a readable and writable storage. The persistent storage may be a non-volatile memory device that does not lose stored instructions and data even after the computer is powered down. In some embodiments, the persistent storage device employs a mass storage device (e.g., magnetic or optical disk, flash memory) as the persistent storage device. In other embodiments, the persistent storage may be a removable storage device (e.g., diskette, optical drive). The system memory may be a read-write memory device or a volatile read-write memory device, such as dynamic random access memory. The system memory may store instructions and data that are required by some or all of the processors at runtime. Furthermore, memory 410 may include any combination of computer-readable storage media including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), magnetic disks, and/or optical disks may also be employed. In some embodiments, memory 410 may include readable and/or writable removable storage devices such as Compact Discs (CDs), digital versatile discs (e.g., DVD-ROMs, dual layer DVD-ROMs), blu-ray discs read only, super-density discs, flash memory cards (e.g., SD cards, min SD cards, micro-SD cards, etc.), magnetic floppy disks, and the like. The computer readable storage medium does not contain a carrier wave or an instantaneous electronic signal transmitted by wireless or wired transmission.

The memory 410 stores executable codes, which when processed by the processor 420, cause the processor 420 to execute the above-mentioned method for processing the abnormal image of the optical machine of the smart glasses.

Furthermore, the method according to the present disclosure may also be implemented as a computer program or computer program product comprising computer program code instructions for performing the above steps defined in the above method of the present disclosure.

Alternatively, the present disclosure may also be implemented as a non-transitory machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) having stored thereon executable code (or computer program, or computer instruction code) that, when executed by a processor of an electronic device (or computing device, server, etc.), causes the processor to perform the steps of the above-described method according to the present disclosure.

Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems and methods according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The foregoing description of the embodiments of the present disclosure has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the various embodiments described. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or the improvement of technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims (10)

1. A method of handling smart eyewear opto-mechanical frame anomalies, comprising:

reading a register representing a parameter of an output picture of the optical machine;

determining that the optical machine has picture abnormality according to the value of the register; and

and executing reset operation on the optical machine.

2. The method of claim 1, wherein,

the registers include a first register, a second register, and/or a third register, wherein

The first register records a check value of the data of the output picture,

a second register records a frame counter value of the output picture or data representing a frame rate of the output picture,

the third register records the resolution value of the output picture.

3. The method of claim 2, wherein,

and reading the value recorded by the first register, and determining that the optical machine has an abnormal picture when the data of the output picture is inconsistent with the data of the picture input to the optical machine according to the check value of the data of the output picture.

4. The method of claim 2, wherein,

and reading the value recorded by the second register, and determining that the optical machine has abnormal picture when the frame counter value or the data representing the frame rate is inconsistent with the corresponding frame count value or data obtained according to the frame rate of the picture input to the optical machine.

5. The method of claim 2, wherein,

and reading the value recorded by the third register, and determining that the optical machine has picture abnormality under the condition that the resolution value is inconsistent with the resolution value of the picture input to the optical machine.

6. The method of claim 2, wherein,

setting the reading sequence of the first register, the second register and the third register, and stopping the subsequent reading of the registers and executing the reset operation on the optical machine under the condition that the optical machine has abnormal pictures according to the value of any one register.

7. The method of claim 1, wherein,

and setting a time period for regularly reading the register according to the electrostatic detection requirement.

8. The method of claim 1, wherein,

performing a reset operation on the light engine includes: and executing power-on and power-off reset operation on the optical machine.

9. A computing device, comprising:

a processor; and

a memory having executable code stored thereon, which when executed by the processor, causes the processor to perform the method of any of claims 1-8.

10. A non-transitory machine-readable storage medium having stored thereon executable code, which when executed by a processor of an electronic device, causes the processor to perform the method of any of claims 1-8.