CN112738325B - Intelligent LED identification method based on Android mobile phone - Google Patents

Abstract

An intelligent LED identification method based on an Android mobile phone comprises the following steps: step 1: initializing the LED lamp bank by adopting a random binary coding algorithm; step 2: shooting through a mobile phone camera to obtain a video; and step 3: intercepting the video every two seconds to obtain a picture set P; and 4, step 4: analyzing each picture Pi by using a YOLOv4 model, and identifying the position coordinates and the on-off states of the LEDs in the picture; and 5: according to the identification result of each time, the LED numbers under the corresponding coordinates are found back; step 6: the front end of the mobile phone forms corresponding light spot coordinates on a front end interface according to the returned coordinates x and y to achieve visualization; and 7: after the user finishes editing and clicks to send, the edited data is analyzed into a data packet form which can be received by the terminal to realize visualization. The method is based on the neural network model trained by a large amount of data and an excellent optimization algorithm, and realizes the identification of the LED lamp group under multiple scenes.

Description

Intelligent LED identification method based on Android mobile phone

Technical Field

The invention relates to the field of intelligent LED illumination, in particular to a method for identifying LED coordinates and serial numbers and editing at a mobile terminal based on video processing.

Background

Along with the layout of a 5G new business state, the intelligent industry has a good development direction and an industry transformation zone, the intelligent lighting single product has a explosive new growth, the product is transformed from single product to multi-product interconnection and intercommunication, which is a new cross-border thought of 'intelligence +', the 'intelligence +' can accelerate the transformation and upgrade of the lighting industry to a high-end, intelligent, green and service direction from multiple aspects, and can boost the high-quality development of the lighting industry, and 'intelligence + LED' is one of branches with deep application prospects.

Along with the development and progress of cities and the continuous improvement of life quality of people, the demand of the night scenes of the cities on the intelligent LED scenes is gradually increased, consumers also have strong interest in adopting the intelligent LED scenes, but few products capable of realizing intelligent control on small colored lamps are available on the market at present. Based on the current situation, the invention is designed and developed.

Disclosure of Invention

In order to overcome the defects of the prior art, the invention provides a method for identifying the coordinates and the serial numbers of the LEDs and editing the LED coordinates and the serial numbers at the mobile terminal based on videos shot by a mobile phone, and meets the requirements of users on more intelligence and high efficiency of the LED lamps.

In order to solve the technical problems, the invention provides the following technical scheme:

an intelligent LED identification method based on an Android mobile phone comprises the following steps:

step 1: initializing the LED lamp bank by adopting a random binary coding algorithm;

step 2: shooting through a mobile phone camera to obtain a video;

and step 3: cutting the video every two seconds to obtain a picture set P { Pi | i ═ 1,2, …, nP };

and 4, step 4: each picture Pi is analyzed using the YOLOv4 model to identify the position coordinates and the on-off status of the LEDs in the picture, as follows:

step 4.1: the conversion from a pixel matrix to a target vector is realized by utilizing the forward calculation of a model, the model adopts multi-target detection, when a first frame of LEDs is fully bright, all LED targets in a picture can be identified, the quantity is M, and each target contains five parameters: { x, y, w, h, p };

step 4.2: in the five parameters, p represents confidence, M LEDs are traversed, the confidence is examined, when p is less than or equal to 0.3, the LEDs are discarded, and K LEDs are remained after one round of screening;

step 4.3: establishing a shaping array C [ K ], initializing the array element value to 0, storing the LED on-off state after each recognition, and establishing a structural body LED, wherein the internal data is as follows: { m, x, y, Ci } for storing the coordinates of the identified LED and the binary code formed after initialization, where m represents the number of the LED;

step 4.4: accurately identifying the LED by using an anti-shake algorithm, preprocessing a reference frame and a current frame of an input video, performing row and column gray projection, and performing cross-correlation operation on data of the gray projection to obtain horizontal shake components, vertical shake components proGk (i) and proGk (j); dividing an image into a plurality of macro blocks, taking the obtained jitter component as the offset of an initial point of block matching motion estimation in each macro block, calculating an SAD value by taking SDSP as a template, and matching to a minimum value point through cross-correlation operation to obtain an optimal matching point; the displacement between the reference point and the best matching point in the reference frame is the block motion vector of the macro block, and the block motion vector of each macro block is calculated by the method, and the block motion vector with the largest occurrence frequency is used as the global motion vector between the reference frame and the current frame to obtain a stable video sequence, so that the aim of accurately positioning the LED coordinates is fulfilled;

step 4.5: when a new round of detection results are stored, when a target of the Cj is detected, the Cj is in a light-on state, the Cj is added with 1, the binary system is represented that the bit is 1, and the Cj is in a light-on state; the undetected LED is not added with 1, which represents that the bit is 0 under binary system and is in a light-off state;

and 5: according to the identification result of each time, the LED numbers under the corresponding coordinates are found back, and the process is as follows:

step 5.1: when the LED is on or off for Y times, the model detects the 01 state of the Y bit and stores the state into an array C, and a corresponding index A [ i ] is searched according to the value stored in the array C [ i ];

step 5.2: after finding, assigning the value stored in the corresponding ai to an integer variable m of the structure LED, namely the serial number of the LED, and then transmitting the LED structure data to the front end;

step 6: the front end of the mobile phone forms corresponding light spot coordinates on the interface of the mobile phone according to the returned coordinates x and y so as to achieve the purpose of visualization;

and 7: after the user finishes editing and clicks to send, the edited data is analyzed into a data packet form which can be received by the terminal so as to realize the purpose of visualization.

Further, the process of step 1) is as follows:

step 1.1: establishing a one-dimensional integer array A [2 x N ] according to the number N of the LEDs, initializing the element value of the array to be 0, wherein the index represents binary coding, the digit of the binary number is determined by an integer variable Y, and the value of Y is obtained by a formula 2^ Y which is more than or equal to N and more than or equal to 2^ Y (Y-1);

step 1.2: random binary coding: firstly, sequentially binary coding is carried out on all LEDs, namely N is traversed, 1 to N are sequentially stored in A [2 x N ], and binary bits are required to be skipped to be a plurality of 0 or 1 continuous coding numerical values during first-time sequential coding;

step 1.3: randomly scrambling all the LED codes by using a fast random scrambling algorithm, wherein the process of scrambling the algorithm is as follows: acquiring the length L of an original array A to be disordered, and setting i to be 0; the random number r starts to be generated in the range of 0-L. Taking the element of the random number position of the array A as the ith element of the new array B, moving the Lth element of the array A to the position of a random number r, subtracting 1 from L, and adding 1 to i; repeating the random number selection process until all elements of the array A are copied into the array B, wherein the array B is the needed disordered array; at this time, the binary number formed by the index of the array B represents the flashing and extinguishing instruction of the LED lamp, and the value stored in the corresponding index is the number of the lamp;

step 1.4: transmitting the flashing rules of the N small colored lamps back to the terminal, adopting two-layer packaging, firstly analyzing the information such as the LED number, the color, the maintenance period and the like transmitted back through the function into RGB specific data of each lamp, then packaging the specific data into a data packet acceptable by the lamp, and finally transmitting the data packet to the terminal; finally, the LED is subjected to color conversion at the frequency of every two seconds; before initialization, firstly setting a first frame as full-bright LED so as to be capable of identifying all small colored lamps in a lens;

step 1.5: when the LED is in a flashing color, the purple light with the shortest wavelength and the smallest diffused halo is used to improve the identification precision, and the purple light brightness within 0-255 is randomly used during training to adapt to identification in different application scenes.

The invention has the beneficial effects that: the method is based on the neural network model trained by a large amount of data and an excellent optimization algorithm, and can realize the identification of the LED lamp group under multiple scenes.

Drawings

FIG. 1 is an overall flow chart of the present invention.

Detailed Description

The following describes in detail a specific embodiment of the method for identifying the LED coordinates and numbers based on video processing and editing at the mobile terminal according to the present invention with reference to the following embodiments.

Referring to fig. 1, an intelligent LED identification method based on an Android mobile phone includes the following steps:

step 1: initializing the LED lamp bank by adopting a random binary coding algorithm, wherein the process is as follows:

step 1.1: and establishing a one-dimensional integer array A [2 x N ] according to the number N of the LEDs, wherein the value of an array element is initialized to 0. The index represents binary code, the digit of the binary number is determined by integer variable Y, and the value of Y is obtained by formula 2^ Y ≧ N ≧ 2^ (Y-1);

step 1.2: random binary coding: all the LEDs are firstly binary coded sequentially, namely N is traversed, and the LEDs from 1 to N are stored in A [2 x N ] sequentially. When the first time of sequential coding is carried out, a plurality of continuous coding numerical values with 0 or 1 in binary digits are required to be skipped, so that the phenomenon that the subsequent identification effect is influenced due to too dark or too bright overall brightness caused by that most LEDs do not emit light or too many LEDs emit light after random scrambling can be avoided;

step 1.3: randomly scrambling all the LED codes by using a fast random scrambling algorithm, wherein the process of scrambling the algorithm is as follows: acquiring the length L of an original array A to be disordered, and setting i to be 0; the random number r starts to be generated in the range of 0-L. Taking the element of the random number position of the array A as the ith element of the new array B, moving the Lth element of the array A to the position of a random number r, subtracting 1 from L, and adding 1 to i; repeating the random number selection process until all elements of the array A are copied into the array B, wherein the array B is the needed disordered array; the whole algorithm is a stable and quick algorithm, the time complexity and the space complexity are both O (n), unstable user experience caused by unstable disorganization in use can be avoided, at the moment, a binary number formed by the index of the array B represents a flashing and extinguishing instruction of the LED lamp, and a value stored in the corresponding index is the number of the lamp;

step 1.4: the flashing and extinguishing rules of the N small colored lamps are transmitted back to the terminal, two-layer packaging is adopted, information such as LED numbers, colors, maintenance periods and the like transmitted back through functions is firstly analyzed into RGB specific data of each lamp, then the specific data are packaged into data packets acceptable by the lamps, and finally the data packets are transmitted to the terminal. Finally, the LED is subjected to color conversion at the frequency of every two seconds; before initialization, firstly setting a first frame as full-bright LED so as to be capable of identifying all small colored lamps in a lens;

step 1.5: when the LED is in a flashing color, the purple light with the shortest wavelength and the smallest diffused halo is used to improve the identification precision, and the purple light brightness within 0-255 is randomly used during training to adapt to identification in different application scenes;

step 2: shooting through a mobile phone camera to obtain a video;

and step 3: intercepting the video every two seconds to obtain a picture set P ═ P_i|i＝1,2,…,n_P}；

And 4, step 4: p for each picture using the Yolov4 model_iAnd analyzing and identifying the position coordinates and the on-off state of the LED in the picture, wherein the process is as follows:

step 4.1: the conversion from a pixel matrix to a target vector is realized by utilizing the forward calculation of a model, the model adopts multi-target detection, when a first frame of LEDs is fully bright, all LED targets in a picture can be identified, the quantity is M, and each target contains five parameters: { x, y, w, h, p };

step 4.2: in the five parameters, p represents confidence, M LEDs are traversed, the confidence is examined, when p is less than or equal to 0.3, the LEDs are discarded, and K LEDs are remained after one round of screening;

step 4.3: establishing a shaping array C [ K ], initializing the array element value to 0, storing the LED on-off state after each recognition, and establishing a structural body LED, wherein the internal data is as follows: { m, x, y, Ci } for storing the coordinates of the identified LED and the binary code formed after initialization, where m represents the number of the LED;

step 4.4: accurately identifying the LED by using an anti-shake algorithm, preprocessing a reference frame and a current frame of an input video, performing row and column gray projection, and performing cross-correlation operation on data of the gray projection to obtain horizontal shake components, vertical shake components proGk (i) and proGk (j); dividing the image into several macro blocks, in each macro block, using the obtained jitter component as the offset of initial point of block matching motion estimation, using SDSP as template to calculate SAD value, and making cross-correlation operation

Matching to the minimum value point to obtain the optimal matching point; the displacement between the reference point and the best matching point in the reference frame is the block motion vector of the macro block, and the block motion vector of each macro block is calculated by the method, and the block motion vector with the largest occurrence frequency is used as the global motion vector between the reference frame and the current frame to obtain a stable video sequence, so that the aim of accurately positioning the LED coordinates is fulfilled;

step 4.5: when a new round of detection results are stored, when a target of the Cj is detected, the Cj is in a light-on state, the Cj is added with 1, the binary system is represented that the bit is 1, and the Cj is in a light-on state; the undetected LED is not added with 1, which represents that the bit is 0 under binary system and is in a light-off state;

and 5: according to the identification result of each time, the LED numbers under the corresponding coordinates are found back, and the process is as follows:

step 5.1: when the LED is on or off for Y times, the model detects the 01 state of the Y bit and stores the state into an array C, and a corresponding index A [ i ] is searched according to the value stored in the array C [ i ];

step 5.2: after finding, assigning the value stored in the corresponding ai to an integer variable m of the structure LED, namely the serial number of the LED, and then transmitting the LED structure data to the front end;

step 6: the front end of the mobile phone forms corresponding light spot coordinates on the interface of the mobile phone according to the returned coordinates x and y so as to achieve the purpose of visualization;

and 7: after the user finishes editing and clicks to send, the edited data is analyzed into a data packet form which can be received by the terminal so as to realize the purpose of visualization.

The embodiments described in this specification are merely illustrative of implementations of the inventive concepts, which are intended for purposes of illustration only. The scope of the present invention should not be construed as being limited to the particular forms set forth in the examples, but rather as being defined by the claims and the equivalents thereof which can occur to those skilled in the art upon consideration of the present inventive concept.

Claims (1)

1. An intelligent LED identification method based on an Android mobile phone is characterized by comprising the following steps:

step 1: initializing the LED lamp bank by adopting a random binary coding algorithm, wherein the process is as follows:

step 1.1: establishing a one-dimensional integer array A [2 x N ] according to the number N of the LEDs, initializing the element value of the array to be 0, wherein the index represents binary coding, the digit of the binary number is determined by an integer variable Y, and the value of Y is obtained by a formula 2^ Y which is more than or equal to N and more than or equal to 2^ Y (Y-1);

step 1.2: random binary coding: firstly, sequentially binary coding is carried out on all LEDs, namely N is traversed, 1 to N are sequentially stored in A [2 x N ], and binary bits are required to be skipped to be a plurality of 0 or 1 continuous coding numerical values during first-time sequential coding;

step 1.3: randomly scrambling all the LED codes by using a fast random scrambling algorithm, wherein the process of scrambling the algorithm is as follows: acquiring the length L of an original array A to be disordered, and setting i to be 0; starting to generate a random number r in the range of 0-L, taking an element at the random number position of the array A as the ith element of the new array B, moving the Lth element of the array A to the position of the random number r, subtracting 1 from L, and adding 1 to i; repeating the random number selection process until all elements of the array A are copied into the array B, wherein the array B is the needed disordered array; at this time, the binary number formed by the index of the array B represents the flashing and extinguishing instruction of the LED lamp, and the value stored in the corresponding index is the number of the lamp;

step 1.4: transmitting the flashing rules of the N small colored lamps back to the terminal, adopting two-layer packaging, firstly analyzing the information such as the LED number, the color, the maintenance period and the like transmitted back through the function into RGB specific data of each lamp, then packaging the specific data into a data packet acceptable by the lamp, and finally transmitting the data packet to the terminal; finally, the LED is subjected to color conversion at the frequency of every two seconds; before initialization, firstly setting a first frame as full-bright LED so as to be capable of identifying all small colored lamps in a lens;

step 1.5: when the LED is in a flashing color, the purple light with the shortest wavelength and the smallest diffused halo is used to improve the identification precision, and the purple light brightness within 0-255 is randomly used during training to adapt to identification in different application scenes;

step 2: shooting through a mobile phone camera to obtain a video;

and step 3: cutting the video every two seconds to obtain a picture set P { Pi | i ═ 1,2, …, nP };

and 4, step 4: each picture Pi is analyzed using the YOLOv4 model to identify the position coordinates and the on-off status of the LEDs in the picture, as follows:

step 4.1: the conversion from a pixel matrix to a target vector is realized by utilizing the forward calculation of a model, the model adopts multi-target detection, when a first frame of LEDs is fully bright, all LED targets in a picture can be identified, the quantity is M, and each target contains five parameters: { x, y, w, h, p };

step 4.2: in the five parameters, p represents confidence, M LEDs are traversed, the confidence is examined, when p is less than or equal to 0.3, the LEDs are discarded, and K LEDs are remained after one round of screening;

step 4.3: establishing a shaping array C [ K ], initializing the array element value to 0, storing the LED on-off state after each recognition, and establishing a structural body LED, wherein the internal data is as follows: { m, x, y, Ci } for storing the coordinates of the identified LED and the binary code formed after initialization, where m represents the number of the LED;

step 4.4: accurately identifying the LED by using an anti-shake algorithm, preprocessing a reference frame and a current frame of an input video, performing row and column gray projection, and performing cross-correlation operation on data of the gray projection to obtain horizontal shake components, vertical shake components proGk (i) and proGk (j); dividing an image into a plurality of macro blocks, taking the obtained jitter component as the offset of an initial point of block matching motion estimation in each macro block, calculating an SAD value by taking SDSP as a template, and matching to a minimum value point through cross-correlation operation to obtain an optimal matching point; the displacement between the reference point and the best matching point in the reference frame is the block motion vector of the macro block, and the block motion vector of each macro block is calculated by the method, and the block motion vector with the largest occurrence frequency is used as the global motion vector between the reference frame and the current frame to obtain a stable video sequence, so that the aim of accurately positioning the LED coordinates is fulfilled;

step 4.5: when a new round of detection results are stored, when a target of the Cj is detected, the Cj is in a light-on state, the Cj is added with 1, the binary system is represented that the bit is 1, and the Cj is in a light-on state; the undetected LED is not added with 1, which represents that the bit is 0 under binary system and is in a light-off state;

and 5: according to the identification result of each time, the LED numbers under the corresponding coordinates are found back, and the process is as follows:

step 5.1: when the LED is on or off for Y times, the model detects the 01 state of the Y bit and stores the state into an array C, and a corresponding index A [ i ] is searched according to the value stored in the array C [ i ];

step 5.2: after finding, assigning the value stored in the corresponding ai to an integer variable m of the structure LED, namely the serial number of the LED, and then transmitting the LED structure data to the front end;

step 6: the front end of the mobile phone forms corresponding light spot coordinates on the interface of the mobile phone according to the returned coordinates x and y so as to achieve the purpose of visualization;

and 7: after the user finishes editing and clicks to send, the edited data is analyzed into a data packet form which can be received by the terminal so as to realize the purpose of visualization.

Images