CN110996020A

CN110996020A - OSD (on-screen display) superposition method and device and electronic equipment

Info

Publication number: CN110996020A
Application number: CN201911283414.9A
Authority: CN
Inventors: 宋炳辉; 周迪
Original assignee: Zhejiang Uniview Technologies Co Ltd
Current assignee: Zhejiang Uniview Technologies Co Ltd
Priority date: 2019-12-13
Filing date: 2019-12-13
Publication date: 2020-04-10
Anticipated expiration: 2039-12-13
Also published as: CN110996020B

Abstract

The application provides an OSD (on screen display) superposition method, an OSD superposition device and electronic equipment, and relates to the technical field of video processing. Firstly, determining an unstallable region in an image frame to be processed, and then determining the region to be superimposed from the image frame according to the attribute information of the OSD to be superimposed and the unstallable region. And the to-be-superposed region and the non-superposable region have no superposed pixel points. And finally, overlapping the OSD to the area to be overlapped. Therefore, the OSD can be automatically superposed in the area outside the area where the key information of the image frame is positioned, so that the problem that the key information is blocked is solved, and the superposition position of the OSD does not need to be manually configured.

Description

OSD (on-screen display) superposition method and device and electronic equipment

Technical Field

The application relates to the technical field of video processing, in particular to an OSD (on screen display) superposition method and device and electronic equipment.

Background

The current applications based On Screen Display (OSD) are very wide, for example, in video surveillance, live network, and the like. Currently, OSD is typically superimposed on a certain area of an image frame of a video, either randomly or fixedly. However, this approach tends to cause the key information of the image frame to be occluded. If the superposition position of the OSD is manually configured, key information cannot be guaranteed not to be shielded;

and in the case of many image frames, the workload of the worker may be very large.

Disclosure of Invention

In view of the above, an object of the present invention is to provide an OSD superimposing method, apparatus and electronic device, which automatically superimposes an OSD on an area other than an area where key information of an image frame is located, so as to solve the problem that the key information is blocked, and do not need to manually configure the OSD superimposing position.

In order to achieve the above purpose, the embodiments of the present application employ the following technical solutions:

in a first aspect, an embodiment of the present application provides an OSD superimposing method, where the method includes:

determining an unstallable region in an image frame to be processed;

determining a region to be superimposed from the image frame according to the attribute information of the OSD to be superimposed and the non-superimposable region, wherein the region to be superimposed and the non-superimposable region have no overlapped pixel points;

and superposing the OSD to the area to be superposed.

In a second aspect, an embodiment of the present application provides an OSD superimposing apparatus, including:

the device comprises a first determining module, a second determining module and a processing module, wherein the first determining module is used for determining an unstallable area in an image frame to be processed;

a second determining module, configured to determine a region to be superimposed from the image frame according to attribute information of an OSD to be superimposed and the non-superimposable region, where the region to be superimposed and the non-superimposable region have no overlapping pixel points;

and the superposition module is used for superposing the OSD to the area to be superposed.

In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, where the memory stores machine executable instructions that can be executed by the processor, and the processor can execute the machine executable instructions to implement the OSD superimposing method according to any one of the foregoing embodiments.

In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements an OSD superimposing method according to any one of the foregoing embodiments.

According to the OSD overlapping method, the device and the electronic equipment provided by the embodiment of the application, when the OSD needs to be overlapped, the non-overlapping area in the image frame to be processed is firstly determined, and then the area to be overlapped is determined from the image frame according to the attribute information of the OSD to be overlapped and the non-overlapping area, wherein the pixel points of the non-overlapping area and the area to be overlapped are not overlapped. And finally, overlapping the OSD to the area to be overlapped. Therefore, the OSD can be selectively superposed on the image frame, thereby solving the problem that the key information in the image frame is blocked by the OSD, and the superposition position of the OSD does not need to be manually configured.

In order to make the aforementioned objects, features and advantages of the present application more comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.

Fig. 1 is a block schematic diagram of an electronic device provided in an embodiment of the present application;

fig. 2 is a schematic flowchart illustrating an OSD superimposing method according to an embodiment of the present application;

FIG. 3 is a breadth first schematic of an embodiment of the present application;

FIG. 4 is a schematic diagram of a coordinate system provided by an embodiment of the present application;

FIG. 5 is a high-priority schematic provided by an embodiment of the present application;

FIG. 6 is a flowchart illustrating the sub-steps included in step S120 of FIG. 2;

FIG. 7 is a schematic diagram of an area expansion provided by an embodiment of the present application;

FIG. 8 is a schematic illustration of a hexagon shape provided by an embodiment of the present application;

FIG. 9 is another flow chart illustrating the sub-steps included in step S120 of FIG. 2;

FIG. 10 is a second schematic diagram illustrating the region expansion provided by the embodiment of the present application;

fig. 11 is a block diagram illustrating an OSD superimposing apparatus according to an embodiment of the present application.

Icon: 100-an electronic device; 110-a memory; 120-a processor; 130-a communication unit; 200-OSD superimposing means; 210-a first determination module; 220-a second determination module; 230-superposition module.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations.

Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.

It is noted that relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

Referring to fig. 1, fig. 1 is a block diagram of an electronic device 100 according to an embodiment of the present disclosure. The electronic device 100 may be, but is not limited to, a server, a video playing device, and the like. The electronic device 100 includes a memory 110, a processor 120, and a communication unit 130. The elements of the memory 110, the processor 120 and the communication unit 130 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, the components may be electrically connected to each other via one or more communication buses or signal lines.

The memory 110 is used to store programs or data. The Memory 110 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an erasable Read-Only Memory (EPROM), an electrically erasable Read-Only Memory (EEPROM), and the like.

The processor 120 is used to read/write data or programs stored in the memory 110 and perform corresponding functions. For example, the memory 110 stores an OSD (On Screen Display) superimposing apparatus 200, and the OSD superimposing apparatus 200 includes at least one software function module that can be stored in the memory 110 in the form of software or firmware (firmware). The processor 120 executes various functional applications and data processing to superimpose the OSD on the region of the extraneous key information by executing software programs and modules stored in the memory 110, such as the OSD superimposing apparatus 200 in the embodiment of the present application.

The communication unit 130 is used for establishing a communication connection between the electronic apparatus 100 and another communication terminal via a network, and for transceiving data via the network.

It should be understood that the structure shown in fig. 1 is only a schematic structural diagram of the electronic device 100, and the electronic device 100 may also include more or fewer components than shown in fig. 1, or have a different configuration than shown in fig. 1. The components shown in fig. 1 may be implemented in hardware, software, or a combination thereof.

Referring to fig. 2, fig. 2 is a flowchart illustrating an OSD superimposing method according to an embodiment of the present application. The method is applied to the electronic device 100. The specific flow of the OSD superimposing method is described in detail below.

In step S110, an unstallable region in the image frame to be processed is determined.

Step S120, determining a region to be superimposed from the image frame according to the attribute information of the OSD to be superimposed and the non-superimposable region.

Step S130, superimpose the OSD on the region to be superimposed.

In this embodiment, the non-superimposable region in the image frame may be determined by manual setting, automatic identification, and the like according to actual needs. The non-superposable region includes key information that cannot be blocked by the OSD, that is, the non-superposable region is a region where the OSD cannot be superposed. The non-superposable region can comprise at least one region, and is determined by actual conditions.

After the non-superposable region is determined, a region to be superposed, on which the OSD can be superposed, may be determined from a region other than the non-superposable region in the image frame according to attribute information of the OSD. And the area to be superposed and the non-superposable area have no superposed pixel points. Then, the OSD may be superimposed to the region to be superimposed.

The attribute information may include a shape, a size, and the like, and may be specifically determined according to actual content of the OSD. For example, if the OSD is a graphic, the attribute information may include a specific shape, size, and the like of the OSD. For example, if the OSD is a circular picture, the attribute information of the OSD may include information such as a circle, a radius, and the like. If the OSD is a word, the attribute information may include a number of words, a shape and a size of each word. In the embodiment of this embodiment, when the OSD is a character, the shape of each character is a rectangle, and the size of each character is the size of the shape of the character, and optionally, the size of each character may be represented by a height requirement and a width requirement.

Therefore, the problem that key information in the image frame is blocked by the OSD can be solved by firstly determining the non-superposable region and then superposing the OSD into the region except the non-superposable region in the image frame, and when the video comprising a plurality of image frames needs to be processed, the superposition position of the OSD does not need to be manually configured frame by frame, and meanwhile, the situation that the OSD is superposed into the region where the key information is located due to carelessness of staff is avoided.

In this embodiment, a preset target included in an image frame to be processed may be identified through an identification algorithm, so as to obtain a first region where the preset target is located, and a non-superimposable region in the image frame is determined according to the first region. For example, as shown in fig. 3, through object recognition, the regions a1, a2, A3, a4 where the preset object is located are identified in the image frame, so that it can be determined that the regions a1, a2, A3, a4 are included in the non-superimposable region.

Optionally, in this embodiment, a preset target may be set in advance according to a service scene corresponding to a video to which the image frame to be processed belongs. For example, the recognition algorithm used can support the recognition of 100 targets, and several classes (for example, 30 classes) can be selected as the preset targets according to the service scenario. When the target is identified, the target capable of being identified does not need to be identified, and only the interested preset target needs to be identified, so that the efficiency of target identification can be improved, and the identification of invalid targets is reduced. For example, in a public safety scenario, instead of setting all identifiable objects as the preset objects, motor vehicles, non-motor vehicles, people (human faces), left-behind objects, and the like may be set as the preset objects.

Optionally, in another embodiment, the electronic device 100 may further determine a second area from the image frame according to the received setting command, and use the second area as the non-superimposable area. The number of the second regions may be one or more. The set command indicates that the video viewer has a fixed non-occlusion requirement for certain areas, which may be areas of interest for the video viewer.

For example, if a video viewer is interested in a picture at the bottom right corner 100 × 100, the region at the bottom right corner 100 × 100 may be set as a fixed non-blocking region, i.e., the second region, so as to avoid blocking the region when the OSD is subsequently superimposed on the region. Therefore, even if the preset target is not included in the area, the key information in the area can not be shielded by the OSD.

Optionally, both the first region determined by the target recognition and the second region determined according to the setting command may be the non-superimposable region. Therefore, the preset target can be prevented from being blocked by the OSD, and the content in the region of interest of the user can be prevented from being blocked.

As shown in fig. 3, if the setting command corresponds to the area a5, the area a5 is the second area. When the first areas are areas a1, a2, A3 and a4, it may be determined that the non-superimposable areas include areas a1, a2, A3, a4 and a5, and the non-superimposable areas include a first area where a preset target is located and a second area corresponding to a setting command. The pixel points in the region to be superimposed and the non-superimposed region are not overlapped, so that the content in the first region and the content in the second region can be ensured not to be shielded by the OSD.

In this embodiment, when the OSD is a graphic, the region to be superimposed may be determined from a region other than the non-superimposable region in the image frame directly according to the shape and size of the OSD.

In this embodiment, when the OSD is a text, the attribute information may include a word count, a height requirement and a width requirement of each text. The area requirement of the OSD can be obtained by calculation according to the height requirement, the width requirement and the word number of each character in the attribute information, and then the area to be superposed is determined according to the area requirement, the width requirement and the height requirement of each character.

Referring to fig. 4, fig. 4 is a schematic diagram of a coordinate system according to an embodiment of the present disclosure. In this embodiment, the coordinate system is a rectangular coordinate system with the top left corner vertex of the image frame as the origin and increasing toward the bottom right corner. Where y in fig. 4 represents a vertical coordinate (i.e., a vertical coordinate value), and x represents a horizontal vertical coordinate (i.e., a horizontal coordinate value). The coordinates of the pixels used in this embodiment adopt the coordinate system shown in fig. 4. It will of course be appreciated that other two-dimensional coordinate systems may be established in other ways.

When the OSD is a character, the region to be superimposed may be calculated in a width-first manner or a height-first manner according to an area requirement, a width requirement, and a height requirement of each character. The breadth-first means that breadth expansion is tried preferentially to obtain the region to be overlapped. As shown in fig. 3, if the height requirements of the characters of the OSD are all h, then h is taken as the height of the superposable region a7, and the superposable region a7 is obtained by horizontal expansion, so as to obtain the region to be superposed according to the superposable region a 7.

High priority means that high expansion is preferentially attempted to obtain the region to be overlapped. As shown in fig. 5, if the width requirements of the characters of the OSD are all w, the w is the width of the superposable region A8, and the superposable region A8 is obtained by vertical expansion, so as to obtain the region to be superposed according to the superposable region A8.

Referring to fig. 6, fig. 6 is a flowchart illustrating sub-steps included in step S120 in fig. 2. Step S120 includes substeps S1211 to substep S1216.

And a substep S1211, obtaining a target height and a target width according to the height requirement and the width requirement of each character.

In the sub-step S1212, a pixel point is selected as an expansion starting point in the image frame.

And a substep S1213 of performing region expansion according to the target height and the target width by using the expansion starting point as a starting point in a width-first manner or a height-first manner to obtain a region to be expanded.

In this embodiment, the OSD characters may have the same height requirement and different width requirements. The target width may be determined according to a width requirement of each letter of the OSD, and the target height may be determined according to a height requirement of each letter. And the target width is the minimum value of the character width requirement in the attribute information of the OSD. Then, any pixel point which is not overlapped with the non-superposable region in the image frame can be selected as an expansion starting point, or any vertex pixel point of the image frame is selected as the expansion starting point, or the top left vertex pixel point of the image frame is selected as the expansion starting point. The above-mentioned determination method of the expansion starting point is only an example, and the expansion starting point may be determined according to actual requirements.

After the expansion starting point is determined, the expansion starting point is used as the starting point through a width priority mode or a height priority mode, and the region is expanded according to the target width and the target height to obtain a region to be expanded. The width of the region to be expanded is the target width, and the height of the region to be expanded is the target height.

And a substep S1214, after obtaining the region to be expanded, determining whether the region to be expanded overlaps with the non-superposable region.

In the sub-step S1215, if the to-be-expanded region and the non-superposable region are not overlapped, the to-be-expanded region is included in the superposable region, and it is determined whether the area of the superposable region meets the area requirement.

And a substep S1216, if the area of the superposable region meets the area requirement, taking the superposable region as the region to be superposed.

After a region to be expanded is obtained, whether the region to be expanded and the non-superposable region are overlapped can be determined through a ray method. In the case of non-overlapping, the region to be expanded may be included in a superimposable region. The superposable area refers to an area without overlapped pixel points with the non-superposable area. Then, whether the area of the superposable region is larger than or equal to the area requirement of the OSD or not can be judged, and if the area of the superposable region is larger than or equal to the area requirement of the OSD, the area of the superposable region can be judged to meet the area requirement of the OSD; otherwise, it may be determined to be not satisfied. If so, stopping the expansion of the region, and taking the superposable region as the region to be superposed. If not, continuing to expand to obtain a new region to be overlapped in the same manner based on the currently obtained stackable region, and then performing substep S1214 again until the stackable region meeting the area requirement cannot be expanded or obtained.

The manner of obtaining the superimposable region by region expansion as described above will be exemplified below with reference to fig. 7, taking a width-first method as an example.

Assuming that the OSD includes 13 characters, the width requirement and the height requirement of all the characters are the same, and the size of each character is 48 × 60 (in pixels). With (0,0) as an expansion starting point, 48 as a target height and 60 as a target width, an area to be expanded D1 is expanded rightward.

And judging whether each pixel point in the region D1 to be expanded is overlapped with the non-superposable regions B1, B2, B3, B4 and B5. The method for judging whether the image frames are overlapped is a ray method, namely, each pixel point in the area D1 to be expanded is respectively used as an original point, and a right ray is drawn until the right boundary of the image frames is reached. If the intersection points of the ray and a certain non-superposable area are odd, the pixel point corresponding to the ray is outside the non-superposable area. On the contrary, if the number of the intersection points is even, the pixel point corresponding to the ray is in the non-superposable area.

After the judgment is completed by the ray method, it is further necessary to judge whether the pixel points in the region D1 to be expanded are on the boundary of the non-superposable region one by one. Thus, it is possible to determine whether the region to be expanded D1 overlaps with the non-superimposable region.

If the region to be expanded D1 does not overlap with the non-superposable region, the region to be expanded D1 may be included in the superposable region B7, and the superposable region B7 includes only one region to be expanded. Since the area of the superimposable region B7 at this time does not satisfy the area requirement of the OSD, the extension is required.

And (48,60) is taken as the top left corner vertex of the new region to be expanded B7, and an area to be expanded D2 is expanded rightwards. The region to be expanded D2 does not overlap with the non-superposable region, and therefore the region to be expanded D2 is included in the superposable region B7. As a result, the superimposition allowable region B7 gradually expands in area. By repeating the above steps, in the case that the new region to be expanded D3 does not overlap with the non-superposable region, the superposable region B7 including the regions to be expanded D1, D2 and D3 can be obtained. Here, the difference between the lateral coordinate values of the adjacent boundaries of the region to be expanded shown in fig. 7 is 1, and the vertical coordinate value of the lower boundary of the region to be expanded D1 plus 1 is the vertical coordinate value of the region to be expanded D5.

Since the area of the superimposable region B7 including the regions D1, D2 and D3 to be expanded still does not satisfy the area requirement, the region D4 to be expanded is obtained by continuing the rightward expansion. Since the region to be expanded D4 overlaps the non-superposable region B3, rightward expansion is stopped. And then, expanding the region to be expanded to the right by (0,48) to obtain a region to be expanded D5, judging whether the region to be expanded D5 is overlapped with the non-superposable region, if not, bringing the region to be expanded D5 into the superposable region B7, and judging whether the updated area of the superposable region B7 meets the area requirement. The superposable area B7 at this time comprises communicated areas D1, D2, D3 and D5 to be expanded. If not, the expansion is continued to the right based on the area to be expanded D5. When the right expansion cannot be continued, a new region to be expanded is obtained by the right expansion with (0,96), and if the new region to be expanded is overlapped with the non-superposable region, the expansion is stopped.

If the width between the right boundary of the region to be expanded D3 and the right boundary of the image frame is smaller than the target width 48, the region to be expanded D5 is obtained by expanding to the right by (0,48) instead of expanding to the right.

If the area of the stackable region obtained when the expansion is stopped still does not meet the area requirement, the expansion starting point can be replaced, and the region expansion is carried out again according to the method to obtain the region to be superposed. Wherein the replaced expansion starting point is not overlapped with the non-superposable area.

In the above example, the vertex pixel point at the upper left corner of the image frame is used as the expansion starting point, and during actual use, the expansion starting point can be selected according to actual requirements, and the expansion direction is determined, and the specific expansion process is modified adaptively.

If a high-priority mode is adopted, when (0,0) is taken as an expansion starting point, the region to be expanded D1 is obtained firstly, and when the region to be expanded D1 is not overlapped with the non-overlapping region and the obtained area of the overlapping region does not meet the area requirement, the region to be expanded D5 is obtained by downward expansion. The height-first mode and the width-first mode have different extension directions, and the description of the height-first mode may refer to the description of the width-first mode, which is not repeated herein.

In another implementation manner of this embodiment, the non-superposable region determined by the first region and the second region is processed into a rectangular region. When the electronic device 100 records the non-superimposable regions, all lines constituting each non-superimposable region are recorded, and the lines are represented by a pair of pairs of coordinate points, and a pair of pixel points represents a line.

Referring to fig. 8, fig. 8 is a schematic view illustrating a hexagon according to an embodiment of the present disclosure. The squaring process is illustrated below with reference to fig. 8.

The hexagon shown in fig. 8 includes 6 vertices, which are P1(x1, y1), P2(x2, y2), P3(x2, y2), P4(x2, y2), P5(x2, y2), and P6(x2, y2) in this order. A rectangular region including a polygonal region can be determined by using (min (x1, x2, x3, x4, x5, x6), min (y1, y2, y3, y4, y5, y6)) as the upper left corner of the rectangle and (max (x1, x2, x3, x4, x5, x6), max (y1, y2, y3, y4, y5, y6)) as the upper right corner of the rectangle. In this way, irregular non-superimposable regions can be treated as standard rectangles. Therefore, the subsequent complex polygon calculation for judging whether the region to be expanded is overlapped with the non-superposable region can be reduced to the standard rectangle calculation, so that the calculation complexity can be greatly reduced, and the calculation speed can be improved.

Referring to fig. 9, fig. 9 is another flow chart illustrating the sub-steps included in step S120 in fig. 2. The non-superposable area is processed into a rectangular area, and the height requirements of all characters are the same. The area to be superimposed can be determined in a width-first manner or a height-first manner. Step S120 may include substeps S1231 to substep S1236.

And a substep S1231, obtaining a target height and a target width according to the height requirement and the width requirement of each character.

And a substep S1232 of selecting a pixel point in the image frame as an expansion starting point.

And a substep S1233 of performing region expansion along a preset expansion direction according to one of the target height and the target width and the expansion starting point to obtain a region to be expanded.

In this embodiment, the height requirements of all the characters are the same, and the height requirement can be directly used as the target height. The target width may be a minimum value in the text width requirement. Then, any pixel point which is not overlapped with the non-superposable region in the image frame can be selected as an expansion starting point, or any vertex pixel point of the image frame is selected as the expansion starting point, or the top left vertex pixel point of the image frame is selected as the expansion starting point. The above-mentioned determination method of the expansion starting point is only an example, and the expansion starting point may be determined according to actual requirements.

After the expansion starting point is determined, area expansion can be performed along a preset expansion direction according to the expansion starting point and the target height, and an area to be expanded is obtained. The area to be expanded is rectangular, the boundary of the area to be expanded, which is perpendicular to the preset expansion direction and is far away from the expansion starting point, is overlapped with one boundary of the image frame, and the height of the area to be expanded is the target height.

Or, performing region expansion along a preset expansion direction according to the expansion starting point and the target width to obtain a region to be expanded. The region to be expanded is rectangular, the boundary of the region to be expanded, which is perpendicular to the preset expansion direction and is far away from the expansion starting point, is overlapped with one boundary of the image frame, and the width of the region to be expanded is the target width.

And the preset expansion direction used when the region is expanded according to the target height is vertical to the preset expansion direction used when the region is expanded according to the target width. For example, with (0,0) as an expansion starting point, the image frame may be expanded rightward to the right boundary of the image frame according to the target height, so as to obtain the region to be expanded, where the preset expansion direction is horizontal rightward. Or, with (0,0) as an expansion starting point, the image frame may be expanded downward to a lower boundary of the image frame according to the target width, so as to obtain the region to be expanded, where the preset expansion direction is vertical downward. The preset expansion direction can be determined according to actual requirements and a set expansion starting point.

And a substep S1234, determining whether the region to be expanded includes a vertex of the non-superposable region.

And a substep S1235, if the region to be expanded does not include the vertex of the non-superposable region, taking the region to be expanded as a superposable region, and judging whether the area of the superposable region meets the area requirement.

And a substep S1236, if the area of the superposable region meets the area requirement, using the superposable region as the region to be superposed.

The area to be expanded and the area which can not be overlapped are both rectangular, and whether the area to be expanded is overlapped with the area which can not be overlapped can be determined by judging whether the area to be expanded comprises the vertex of the area which can not be overlapped. When the area to be expanded comprises the vertex of the non-superposable area, judging that the area to be expanded is overlapped with the non-superposable area. When the area to be expanded does not comprise the vertex of the non-superposable area, judging that the area to be expanded and the non-superposable area are not overlapped. When the areas are not overlapped, the area to be expanded can be used as an overlapped area. And then judging whether the area of the superposable region is larger than or equal to the area requirement of the OSD. If the size of the superposable region is larger than or equal to the size of the superposable region, the superposable region can be judged to meet the area requirement. If so, determining that the area of the superposable region does not meet the area requirement. And when the area of the superposable region meets the area requirement, stopping the region expansion, and taking the superposable region as the region to be superposed.

Referring again to fig. 9, step S120 may further include a sub-step S1237. When the region to be expanded includes the vertex of the non-superposable region, performing substep S1237.

And a substep S1237, reducing the region to be expanded along a direction opposite to the preset expansion direction according to the vertex of the non-superposable region, so as to obtain an updated region to be expanded.

The current region to be expanded includes a vertex of the non-superposable region, and the superposable region does not overlap with the non-superposable region, so that the current region to be superposed needs to be adjusted and updated. Optionally, the area to be expanded may be reduced along the same direction as the preset expansion direction of the current area to be expanded obtained by expansion, so as to obtain an updated area to be expanded. And the updated region to be expanded does not comprise the vertex of the non-superposable region. Then, sub-step S1235 is performed.

Referring again to fig. 9, step S120 may further include a sub-step S1238. When the area of the superposable region does not satisfy the area requirement, substep S1238 is performed.

And a substep S1238 of expanding the superposable region according to the target height or the target width along a direction perpendicular to the preset expansion direction to obtain an updated region to be expanded.

The area of the superimposable region does not satisfy the area requirement, and indicates that the area of the superimposable region at this time is smaller than the area requirement, and therefore, it is necessary to continue the region expansion. And at the moment, the superimposable region can be expanded continuously along the direction vertical to the preset direction according to the target height so as to obtain an updated region to be expanded. And the height of the updated region to be expanded is integral multiple of the target height.

If the region expansion is performed according to the target width initially, the superimposable region at this time may continue to be expanded according to the target width along a direction perpendicular to the preset direction, so as to obtain an updated region to be expanded. And the updated width of the area to be expanded is integral multiple of the target width.

After the updated region to be extended is obtained, the substep S1234 may be continuously performed.

If the area of the superposable region obtained when the expansion is stopped still cannot meet the area requirement, the expansion starting point can be replaced according to the historical expansion starting point and the vertex of the non-superposable region, and the region expansion is carried out again to obtain the region to be superposed. And the replaced expansion starting point is not overlapped with the vertex of the non-superposable area.

And determining to stop the expansion when the obtained superposable region cannot be subjected to region expansion under the condition of not modifying the expansion starting point. For example, with (0,0) as an expansion starting point, expanding to the right of the image frame to obtain an area to be expanded, and when the area to be expanded does not include a vertex of the non-superposable area, taking the area to be expanded as the superposable area. If the area of the stackable region does not meet the area requirement, the lower boundary of the stackable region is stretched downwards to increase the height of the stackable region. If the height of the image frame is less than 2 times the target height, it may be determined that the stretching cannot be performed within the image frame, at which point the expanding may be stopped.

Optionally, in an implementation manner of this embodiment, the performing, according to the target height and the expansion starting point, region expansion along a preset expansion direction includes: and performing region expansion to the right according to the target height by taking the top left corner vertex of the image frame as the expansion starting point.

In this case, the expansion starting point may be replaced according to the following manner.

And determining a lower target boundary from the non-superposable region according to the longitudinal coordinate value of the last expansion starting point, and determining the longitudinal coordinate value of the replaced expansion starting point according to the longitudinal coordinate value of the lower target boundary. And the longitudinal coordinate value of the target lower boundary is greater than and closest to the longitudinal coordinate value of the last expansion starting point. Alternatively, the vertical coordinate value of the target lower boundary may be added by 1 to serve as the vertical coordinate value of the updated expansion start point.

And determining a target right boundary from the non-superposable region according to the transverse coordinate value of the last expansion starting point, and determining the transverse coordinate value of the replaced expansion starting point according to the transverse coordinate value of the target right boundary. And the transverse coordinate value of the target right boundary is greater than and closest to the transverse coordinate value of the last expansion starting point. Alternatively, the horizontal coordinate value of the target right boundary may be added by 1 to serve as the horizontal coordinate value of the updated extension start point.

If the extension direction or the coordinate system used is different, the determination method of the updated coordinates of the extension start point may be adaptively modified.

The method shown in fig. 9 will be described below with reference to fig. 10 by taking a width-first method as an example.

Assuming that the OSD is "this is an OSD to be superimposed", the font size is 48 × 48, and the number of characters is 13, the area requirement of the OSD is calculated as: 48 × 13 ═ 29952 square pixels. When superimposing words, a word pitch may also be set, where the default word pitch is 0 for ease of understanding. The image frame has a size of 1920 × 1080 (where the width is 1920 and the height is 1080).

And (x, y) is taken as an expansion starting point (x ═ y ═ 0 initially), and (1920,47) is taken as an end point, so that the region to be expanded is obtained. And then judging whether the area to be expanded comprises the vertex of the non-superposable area. If yes, reducing the area to be expanded to the left according to the included vertex of the non-superposable area. For example, if the region to be expanded includes vertices of a plurality of non-superposable regions, and the vertex at the top left corner of the vertices is (a, b), then a-1 may be used as the updated lateral coordinate value of the region to be expanded. The updated region to be expanded does not include the vertices of the non-superposable region, and the updated region to be expanded can be represented as the superposable region C7 in fig. 10. The area of the superposable area C7 is 48 a, and it is determined whether 48 a is greater than or equal to the area requirement 29952. If the sum is greater than or equal to the preset value, the superposable region C7 can be determined as the region to be superposed.

If the area of the superposable region C7 does not satisfy the area requirement, the height of the superposable region C7 may be increased by 48 to increase the superposable region to obtain a new region to be expanded, and the above steps may be repeated until the region to be superposed is obtained or it is determined that the region to be superposed cannot be obtained.

When it is determined that the region to be superimposed cannot be obtained, a new expansion starting point (x ', y') can be selected as the top left corner vertex of the superimposable region, and then the superimposable region is searched again according to the above process. Optionally, the new extended starting point is selected as follows.

Firstly, screening out a pair of coordinate points corresponding to the lower boundary of all the non-superposable regions, then judging whether the longitudinal coordinate value of each pair of coordinate points is greater than the longitudinal coordinate value of the last expansion starting point, and calculating to obtain the difference value between the longitudinal coordinate value of each pair of coordinate points and the longitudinal coordinate value of the last expansion starting point. And then determining a pair of pixel points with the minimum difference and the longitudinal coordinate value larger than the longitudinal coordinate value of the last expansion starting point. Finally, the vertical coordinate value of the pair of pixel points is added with 1 to be used as the vertical coordinate value y' of the new expansion starting point.

Firstly, screening a pair of coordinate points corresponding to the right boundary of all the non-superposable regions, then judging whether the transverse coordinate value of each pair of coordinate points is larger than that of the last expansion point, and calculating to obtain the difference value between the transverse coordinate value of each pair of coordinate points and that of the last expansion starting point. And then determining a pair of pixel points with the minimum difference value and the transverse coordinate value larger than the transverse coordinate value of the last expansion starting point. Finally, the horizontal coordinate value of the pair of pixel points is added with 1 to be used as the horizontal coordinate value x' of the new expansion starting point.

The following exemplifies the selection rule of the new extension starting point. The last extension starting point is (x, y).

And searching the lower boundary of one non-superposable region in all the non-superposable regions, and assuming that two coordinate points of the lower boundary are respectively (w1, h1), (w2, h1), h1> y and the value of h1-y is minimum, taking h1+1 as the longitudinal coordinate value y' of the new expansion starting point.

And searching the right boundary of one non-superposable region in all the non-superposable regions, and assuming that two coordinate points of the right boundary are respectively (w3, h3), (w3, h4), w3> x and the value of w3-x is minimum, taking w3+1 as the lateral coordinate value x' of the new expansion starting point.

In this embodiment, after the region to be superimposed is determined, the OSD may be superimposed on the region to be superimposed according to the OSD setting parameter. Wherein, the setting parameters may include font size, color, font type, etc.

In this embodiment, in the process of searching for the region to be superimposed, the area and position information of different superimposable regions may be recorded. If the region to be superimposed that meets the area requirement is not found by the method corresponding to fig. 6 or fig. 8, the OSD may be segmented or truncated according to a preset policy, the recorded area of the superimposable region, and the recorded position information. And then overlapping the cut or cut OSD to the corresponding overlapping area. The segmentation refers to dividing one piece of data into multiple pieces to be superposed. The segmentation can be realized by adopting an equal segmentation method, semantic analysis and the like. And the truncation refers to taking the superposable region with the largest area as the region to be superposed, then sequentially superposing the contents to be superposed to the region to be superposed, and discarding the rest contents which cannot be superposed.

In order to perform the corresponding steps in the above embodiments and various possible manners, an implementation manner of the OSD superimposing apparatus 200 is given below, and optionally, the OSD superimposing apparatus 200 may adopt the device structure of the electronic device 100 shown in fig. 1. Further, referring to fig. 11, fig. 11 is a block diagram illustrating an OSD superimposing apparatus 200 according to an embodiment of the present application. It should be noted that the basic principle and the technical effects of the OSD superimposing apparatus 200 provided in the present embodiment are the same as those of the above embodiment, and for the sake of brief description, no part of the present embodiment is mentioned, and corresponding contents in the above embodiment may be referred to. The OSD superimposing apparatus 200 may include: a first determination module 210, a second determination module 220, and a superposition module 230.

The first determining module 210 is configured to determine an unstallable region in an image frame to be processed.

The second determining module 220 is configured to determine a region to be superimposed from the image frame according to attribute information of an OSD to be superimposed and the non-superimposable region, where the region to be superimposed and the non-superimposable region have no overlapping pixel points.

The superimposing module 230 is configured to superimpose the OSD onto the region to be superimposed.

Alternatively, the modules may be stored in the memory 110 shown in fig. 1 in the form of software or Firmware (Firmware) or be fixed in an Operating System (OS) of the electronic device 100, and may be executed by the processor 120 in fig. 1. Meanwhile, data, codes of programs, and the like required to execute the above-described modules may be stored in the memory 110.

An embodiment of the present application further provides a readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the OSD superimposing method is implemented.

To sum up, the embodiment of the present application provides an OSD superimposing method, an OSD superimposing apparatus, and an electronic device, when an OSD needs to be superimposed, first a non-superimposable region in an image frame to be processed is determined, and then a region to be superimposed is determined from the image frame according to attribute information of the OSD to be superimposed and the non-superimposable region, where pixel points of the region to be superimposed and the non-superimposable region do not overlap. And finally, overlapping the OSD to the area to be overlapped. Therefore, the OSD can be selectively superposed on the image frame, thereby solving the problem that the key information in the image frame is blocked by the OSD, and the superposition position of the OSD does not need to be manually configured.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An OSD superimposing method, the method comprising:

determining an unstallable region in an image frame to be processed;

and superposing the OSD to the area to be superposed.

2. The OSD superimposing method according to claim 1, wherein the OSD is text, the attribute information includes a number of words, a height requirement and a width requirement of each text, and the determining the region to be superimposed from the image frame according to the attribute information of the OSD to be superimposed and the non-superimposable region includes:

and determining the region to be superposed from the image frame according to an area requirement, and the height requirement and the width requirement of each character, wherein the area requirement is calculated according to the attribute information.

3. The OSD superimposing method of claim 2, wherein the non-superimposable region is processed as a rectangular region, the height requirement of all the characters is the same, and the determining the region to be superimposed from the image frame according to the area requirement, the height requirement of each character and the width requirement comprises:

obtaining a target height and a target width according to the height requirement and the width requirement of each character, wherein the target width is the minimum value in the character width requirements;

selecting a pixel point in the image frame as an expansion starting point;

performing area expansion along a preset expansion direction according to one of the target height and the target width and the expansion starting point to obtain an area to be expanded, wherein the area to be expanded is rectangular, a boundary of the area to be expanded, which is perpendicular to the preset expansion direction and is far away from the expansion starting point, is overlapped with one boundary of the image frame, and the height of the area to be expanded is the target height or the width is the target width;

judging whether the area to be expanded comprises a vertex of the non-superposable area;

if the to-be-expanded region does not comprise the vertex of the non-superposable region, taking the to-be-expanded region as a superposable region, and judging whether the area of the superposable region meets the area requirement or not;

and if the area of the superposable region meets the area requirement, taking the superposable region as the region to be superposed.

4. The OSD superimposing method of claim 3, wherein the determining the region to be superimposed from the image frame according to an area requirement, a height requirement and a width requirement of each text further comprises:

if the to-be-expanded region comprises the vertex of the non-superposable region, reducing the to-be-expanded region along the direction opposite to the preset expansion direction according to the vertex of the non-superposable region to obtain an updated to-be-expanded region, and executing a step of judging whether the area of the superposable region meets the area requirement, wherein the updated to-be-expanded region does not comprise the vertex of the non-superposable region;

if the area of the stackable region does not meet the area requirement, expanding the stackable region along a direction perpendicular to the preset expansion direction according to the target height or the target width to obtain an updated region to be expanded, and executing a step of judging whether the region to be expanded comprises a vertex of the non-stackable region.

5. The OSD superimposing method of claim 3, wherein the determining the region to be superimposed from the image frame according to an area requirement, a height requirement and a width requirement of each text further comprises:

and if the area of the superposable region obtained when the expansion is stopped does not meet the area requirement, replacing the expansion starting point according to the historical expansion starting point and the vertex of the non-superposable region, and performing region expansion again to obtain the region to be superposed, wherein the replaced expansion starting point is not overlapped with the vertex of the non-superposable region.

6. The OSD superimposing method according to claim 5, wherein the OSD superimposing method further includes,

the expanding the region along a preset expanding direction according to the target height and the expanding starting point comprises the following steps:

taking the top left corner vertex of the image frame as the expansion starting point, and performing region expansion to the right according to the target height;

the replacing the expansion starting point according to the history expansion starting point and the vertex of the non-superposable area comprises the following steps:

determining a lower target boundary from the non-superposable region according to a longitudinal coordinate value of a last expansion starting point, and determining a longitudinal coordinate value of a replaced expansion starting point according to the longitudinal coordinate value of the lower target boundary, wherein the longitudinal coordinate value of the lower target boundary is greater than and closest to the longitudinal coordinate value of the last expansion starting point;

and determining a target right boundary from the non-superposable region according to the transverse coordinate value of the last expansion starting point, and determining the transverse coordinate value of the replaced expansion starting point according to the transverse coordinate value of the target right boundary, wherein the transverse coordinate value of the target right boundary is greater than and closest to the transverse coordinate value of the last expansion starting point.

7. The OSD superimposing method of claim 2, wherein the determining the region to be superimposed from the image frame according to the area requirement, the height requirement and the width requirement of each text comprises:

selecting a pixel point from the image frame as an expansion starting point;

performing area expansion according to the target height and the target width by taking the expansion starting point as a starting point in a width-first mode or a height-first mode to obtain an area to be expanded, wherein the width of the area to be expanded is the target width, and the height of the area to be expanded is the target height;

after the area to be expanded is obtained, judging whether the area to be expanded is overlapped with the non-superposable area;

if the to-be-expanded region is not overlapped with the non-superposable region, bringing the to-be-expanded region into a superposable region, and judging whether the area of the superposable region meets the area requirement, wherein the superposable region comprises one to-be-expanded region or a plurality of communicated to-be-expanded regions;

if the area of the superposable region meets the area requirement, taking the superposable region as the region to be superposed;

if the area of the stackable region does not meet the area requirement, continuing to expand;

and if the area of the stackable region obtained when the expansion is stopped does not meet the area requirement, replacing the expansion starting point, and performing region expansion again to obtain the region to be superposed, wherein the replaced expansion starting point is not overlapped with the non-stackable region.

8. The OSD superimposing method of claim 1, wherein the determining the non-superimposable region in the image frame comprises:

performing target recognition on the image frame according to a preset target, and taking a first area where the preset target is located as the non-superposable area; and/or the presence of a gas in the gas,

and taking the second area determined according to the received setting command as the non-superimposable area.

9. An OSD superimposing apparatus, comprising:

10. An electronic device comprising a processor and a memory, the memory storing machine executable instructions executable by the processor to implement the OSD superimposition method of any one of claims 1-8.