CN115393557A - Method, apparatus, device and medium for identifying region of object in image - Google Patents

Abstract

Methods, apparatuses, devices, and media for identifying regions of objects in an image are provided. In one method, at least one identified region of at least one identified object in an image is acquired. An action for specifying a target region of a target object in an image is detected. The target area of the target object specified by the action is updated based on the at least one identified area. The current target area may be updated based on the respective identified areas to indicate the target object in a single area without performing area splitting.

Description

Method, apparatus, device and medium for identifying region of object in image

Technical Field

Example implementations of the present disclosure relate generally to image processing, and more particularly, to methods, apparatuses, devices, and computer-readable storage media for identifying regions of objects in images in panoramic panning applications.

Background

With the development of digitization technology, a variety of panoramic roaming applications have been developed. The user may roam in a virtual scene of the panoramic roaming application to experience an immersive visual effect. When a user clicks on various objects (e.g., walls, ceilings, etc.) in a virtual scene, the panoramic roaming application may provide more information about the clicked object. This requires the producer to identify in advance the regions of each object in the image. At this time, how to identify the region of the object in the image in a more convenient and efficient manner becomes an urgent problem to be solved.

Disclosure of Invention

In a first aspect of the disclosure, a method for identifying a region of an object in an image is provided. In the method, at least one identified region of at least one identified object in an image is acquired. An action for specifying a target region of a target object in an image is detected. The target area of the target object specified by the action is updated based on the at least one identified area.

In a second aspect of the disclosure, an apparatus for identifying a region of an object in an image is provided. The device includes: an acquisition module configured to acquire at least one identified region of at least one identified object in an image; a detection module configured to detect an action for specifying a target area of a target object in an image; and an update module configured to update a target region of the target object specified by the action based on the at least one identified region.

In a third aspect of the disclosure, an electronic device is provided. The electronic device includes: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions when executed by the at least one processing unit cause the apparatus to perform a method according to the first aspect of the disclosure.

In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, having stored thereon a computer program, which, when executed by a processor, causes the processor to carry out the method according to the first aspect of the present disclosure.

It should be understood that what is described in this section is not intended to limit key features or essential features of implementations of the disclosure, nor is it intended to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The above and other features, advantages and aspects of various implementations of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings. In the drawings, like or similar reference characters designate like or similar elements, and wherein:

FIG. 1A illustrates a block diagram of an example environment in which implementations of the present disclosure can be implemented;

FIG. 1B illustrates a block diagram for identifying regions of an object in an image, in accordance with some aspects;

FIG. 2 illustrates a block diagram of a region for identifying an object in an image, according to some implementations of the present disclosure;

FIG. 3 illustrates a block diagram for describing regions in an image, according to some implementations of the present disclosure.

FIG. 4 illustrates a block diagram of representing regions of an object in an image with polygons in accordance with some implementations of the present disclosure;

FIG. 5 illustrates a block diagram of representing regions of an object in an image with polygons in accordance with some implementations of the present disclosure;

FIG. 6 illustrates a block diagram of a process for determining a union of a plurality of identified regions in accordance with some implementations of the present disclosure;

FIG. 7 illustrates a block diagram of a process for determining a union of a plurality of identified regions in accordance with some implementations of the present disclosure;

FIG. 8 illustrates a block diagram for determining whether multiple contour points of a target region lie in the same plane, according to some implementations of the present disclosure;

FIG. 9 illustrates a flow diagram of a method for identifying regions of objects in an image, according to some implementations of the present disclosure;

FIG. 10 illustrates a block diagram of an apparatus for identifying regions of objects in an image, in accordance with some implementations of the present disclosure; and

fig. 11 illustrates a block diagram of a device capable of implementing various implementations of the present disclosure.

Detailed Description

Implementations of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain implementations of the present disclosure are illustrated in the accompanying drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the implementations set forth herein, but rather are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and implementations of the present disclosure are for illustration purposes only and are not intended to limit the scope of the present disclosure.

In describing implementations of the present disclosure, the terms "include," including, "and their like are to be construed as being inclusive, i.e.," including, but not limited to. The term "based on" should be understood as "based at least in part on". The term "one implementation" or "the implementation" should be understood as "at least one implementation". The term "some implementations" should be understood as "at least some implementations". Other explicit and implicit definitions are also possible below. As used herein, the term "model" may represent an associative relationship between various data. For example, the above-described association may be obtained based on various technical solutions that are currently known and/or will be developed in the future.

It will be appreciated that the data involved in the subject technology, including but not limited to the data itself, the acquisition or use of the data, should comply with the requirements of the corresponding laws and regulations and related regulations.

It is understood that before the technical solutions disclosed in the embodiments of the present disclosure are used, the user should be informed of the type, the use range, the use scene, etc. of the personal information related to the present disclosure and obtain the authorization of the user through an appropriate manner according to the relevant laws and regulations.

For example, in response to receiving an active request from a user, a prompt message is sent to the user to explicitly prompt the user that the requested operation to be performed would require the acquisition and use of personal information to the user. Thus, the user can autonomously select whether to provide personal information to software or hardware such as an electronic device, an application program, a server, or a storage medium that performs the operations of the technical solution of the present disclosure, according to the prompt information.

As an optional but non-limiting implementation manner, in response to receiving an active request from the user, the prompt information is sent to the user, for example, a pop-up window may be used, and the prompt information may be presented in text in the pop-up window. In addition, a selection control for providing personal information to the electronic equipment by the user's selection of "agree" or "disagree" can be carried in the pop-up window.

It is understood that the above notification and user authorization process is only illustrative and not limiting, and other ways of satisfying relevant laws and regulations may be applied to the implementation of the present disclosure.

Example Environment

In order to provide more information about objects in a panoramic navigation application, a producer needs to capture images of real scenes in advance and identify the areas in which the objects in the images are located using the panoramic navigation production application. An application environment in accordance with one exemplary implementation of the present disclosure is described with reference to fig. 1A, which illustrates a block diagram 100A of an example environment in which implementations of the present disclosure can be implemented. As shown in fig. 1, in a panoramic roaming production application, an image 110 of a real scene may be received. Hereinafter, only a panoramic image will be taken as an example of an image to be processed. Alternatively and/or additionally, the image to be processed may also include an acquired image (e.g., an acquired top view, bottom view, front view, back view, left view, right view) for generating a panoramic image.

As shown in FIG. 1, the image 110 may include multiple types of objects, such as a wall 120, a television 130 on the wall 120, a ceiling 140, a floor 150, and so forth. It has been proposed to identify regions of objects in an image using a selection tool or the like. For example, a user of the panoramic navigation production tool (i.e., a production person) may manually select a plurality of points in an image and identify an area having the plurality of points as outline points as an area of an object. At this point, the fabricator can manually enter a plurality of points in the image 110 to identify an area of an object. Alternatively and/or additionally, the producer may determine the region of the object using other operations, such as box selection.

FIG. 1B illustrates a block diagram 100B for identifying regions of objects in an image, according to some aspects. As shown in fig. 1B, the user may frame a region 132 in the image 110 in which the television 130 is located and identify the region 132 as being of the type "television". Further, the user may select a plurality of points (e.g., contour points 170, 172, etc.) at the contour of the wall 120 in the image 110 in order to determine the area where the wall 120 is located.

Because the television 130 is present on the wall 120, which results in the associated area 132 of the television 130 also being included within the area of the wall 120, the fabricator cannot use a single area to represent the area of the wall 120. At this time, the maker has to split the area where the wall 120 is located into two or more areas. For example, a concave area 124, as shown by legend 160 "wall," may be identified first, and then an unidentified area 122 in the wall 120 may be identified (as shown by legend 162). At this time, although the wall 120 is the same object in the real environment, it needs to be identified with two areas in the virtual scene.

It will be appreciated that fig. 1A and 1B show only a simple example with the wall 120 and television 130 as examples, and that if an object involves more complex situations, the object may need to be broken into more regions. The excessive area increases the workload of the manufacturer and may cause complication of the subsequent processing. In this case, how to identify the region of the object in the image in a simpler and more efficient manner becomes an urgent problem to be solved.

Summary procedure for image identification

In order to solve the deficiencies in the above technical solutions, according to an exemplary implementation of the present disclosure, a technical solution for identifying a region of an object in an image is proposed. An overview of identifying an image is described with reference to fig. 2, which fig. 2 shows a block diagram 200 for identifying regions of an object in an image, according to some implementations of the present disclosure. As shown in FIG. 2, an identified region 132 in image 110 may be first acquired, where identified region 132 may be the region identified by the fabricator in the previous step. For example, the producer may frame the television 130 in the image 110 to determine the region 132. At this point, the identified region 132 may be received directly.

The producer may process each object in the image 110 one by one, for example, the producer may select a plurality of contour points of the wall 120 in the image 110 in order to specify a target area in which the wall 120 in the image 110 is located. At this time, the target area may be determined based on the action of the maker. In FIG. 2, the target area determined based on the actions of the producer may be area 210. In the initial stage, the target area may include the full extent within the outline of the area 210 (i.e., including the area 132 in which the television is located).

Further, the target area of the wall specified by the action may be updated based on the identified area 132. In other words, the region 132 may be removed from the region 210, at which point the updated target region will include a ring shape, i.e., including only the wall portion as shown by the legend 160 but not the television portion as shown by the legend 164. With the exemplary implementation of the present disclosure, the area where the television is located can be represented with a single annular shape. In this way, on the one hand, additional operations of the manufacturer to manually split the wall area can be avoided. On the other hand, a plurality of areas are not needed to identify the same object, so that the association relationship between the object and the areas can be described in a one-to-one correspondence manner, and the complexity of subsequent processing is reduced.

Detailed procedure for image identification

Having described the outline of the image markers with reference to fig. 2, more details regarding the image markers will be described below. According to one exemplary implementation of the present disclosure, it may be first determined whether there is overlap between the identified region and the currently identified target region. It will be appreciated that the need for splitting is only possible if there is overlap between the two regions. In this way, it can be determined in advance whether the image identification method according to one exemplary implementation of the present disclosure needs to be executed, and the method according to one exemplary implementation of the present disclosure can be called only when needed.

According to an example implementation of the present disclosure, if it is determined that the at least one identified region is located within the target region, an updated target region may be determined based on a difference between the target region and the at least one identified region. In particular, the difference set between the target region and the identified region (i.e., the portion that is within the target region but not within the identified region) may be taken as the updated target region. In the following, further details of an exemplary implementation according to the present disclosure are described with reference to fig. 3. Fig. 3 illustrates a block diagram 300 for describing regions in an image, according to some implementations of the present disclosure.

As shown in fig. 3, a region may be represented by a sequence of contour points located at the contour of the region. Assume that the contour of region 210 includes n1 contour points (x) ₁₁ ,y ₁₁ )、(x ₁₂ ,y ₁₂ )、…、(x _1n1 ,y _1n1 ) Then the region 210 may be represented by a sequence of contour points 310: (x) ₁₁ ,y ₁₁ ),(x ₁₂ ,y ₁₂ ),…,(x _1n1 ,y _1n1 ). For another example, region 132 may be represented by a sequence of contour points 320: (x) ₂₁ ,y ₂₁ ),(x ₂₂ ,y ₂₂ ),…,(x _2n2 ,y _2n2 ) In this case, n1 and n2 are positive integers, respectively.

According to one exemplary implementation of the present disclosure, the updated target region may be represented by polygon data. The polygon data may be defined based on a variety of formats. For example, the updated target region may be defined by a sequence of contour points of the outer contour of the target region and a sequence of contour points of the outer contour of the at least one identified region. In particular, the updated annular target region may be defined based on the two sequences of contour points described above.

For further details regarding polygon data, reference is made to FIG. 4, which illustrates a block diagram 400 for representing regions of objects in an image using polygons in accordance with some implementations of the present disclosure. As shown in FIG. 4, polygon 410 may be defined based on sequence of contour points 310 for the outline of region 210 and sequence of contour points 320 for the outline of identified region 132:

polygon = { outer contour: [ (x) ₁₁ ,y ₁₁ ),(x ₁₂ ,y ₁₂ ),…,(x _1n1 ,y _1n1 )]And excluding the area: [ (x) ₂₁ ,y ₂₁ ),(x ₂₂ ,y ₂₂ ),…,(x _2n2 ,y _2n2 )]}

According to one exemplary implementation of the present disclosure, a polygon may include two parts: an outer contour representing an outer boundary of the polygon; and an exclusion region indicating a region inside the polygon that should be excluded. With the exemplary implementations of the present disclosure, a region that includes one or more "holes" (i.e., excluded regions) may be represented with a single polygon data structure. In this way, regions of objects in an image may be identified in a richer shape. Further, the exclusion area is automatically determined based on the identified area, thereby avoiding the manual operation of selecting the contour point of the object to be identified by a manufacturer in the prior art.

Although fig. 4 above describes an example of updating the target region based on the difference set between the target region and the identified region with a single identified region 132 as an example. Alternatively and/or additionally, there may be multiple identified regions within the target region, in which case the exclusion region may be defined using a sequence of contour points of the outer contour of the multiple identified regions. In other words, the exclusion area may include a plurality of sequences of contour points representing the outer contour of the area. The case where there are a plurality of identified regions is described with reference to fig. 5.

Fig. 5 illustrates a block diagram 500 of representing regions of an object in an image with polygons according to some implementations of the present disclosure. As shown in fig. 5, there may be regions 132 and 510 that have been identified. For example, region 132 may correspond to a television type object and region 510 may correspond to a stereo type object. At this time, the contour points of the regions 132 and 510 are defined by the contour point sequences 320 and 520, respectively. As shown in fig. 5, the producer may identify in the image an area 210 where the wall 120 is located, and the contour points of the area 210 are defined by a sequence of contour points 310. At this time, the outline of the polygon 530 may be represented as the sequence of outline points 310, and the exclusion area may be represented as the sequences of outline points 320 and 520. Specifically, polygon 530 may be represented as:

polygon = &

Outer contour: [ (x) ₁₁ ,y ₁₁ ),(x ₁₂ ,y ₁₂ ),…,(x _1n1 ,y _1n1 )]，

An exclusion area: [ (x) ₂₁ ,y ₂₁ ),(x ₂₂ ,y ₂₂ ),…,(x _2n2 ,y _2n2 )]，[(x ₃₁ ,y ₃₁ ),(x ₃₂ ,y ₃₂ ),…,(x _3n3 ,y _3n3 )]}

As the maker continues to identify regions of various objects in the image, there will be more and more identified regions. As the number of identified regions increases, the difference between each newly added region and the identified region needs to be determined, which can present challenges to computational accuracy while reducing efficiency. Assume that a plurality of areas A, B, C and D are currently identified and if an area E is currently newly added, it needs to be calculated separately: e = E-A, E = E-B, E = E-C, E = E-D. The greater the number of regions, the higher the computational cost. At this time, if the difference set between the target region and each identified region is directly determined, a huge computational resource overhead may be incurred.

According to one exemplary implementation of the present disclosure, a union of the respective identified regions may be determined, and the union may be taken as an exclusion region, so as to simplify the complexity of the post-processing. For the example in fig. 5, the polygon 530 may be represented as:

polygon = &

Outer contour: [ (x) ₁₁ ,y ₁₁ ),(x ₁₂ ,y ₁₂ ),…,(x _1n1 ,y _1n1 )]，

An exclusion area: [ (x) ₂₁ ,y ₂₁ ),(x ₂₂ ,y ₂₂ ),…,(x _2n2 ,y _2n2 )]∪[(x ₃₁ ,y ₃₁ ),(x ₃₂ ,y ₃₂ ),…,(x _3n3 ,y _3n3 )]}

Then, an updated target region may be determined based on a difference set between the target region and the union. More details of determining a union are described below with reference to fig. 6, which fig. 6 illustrates a block diagram 600 of a process for determining a union of multiple identified regions, in accordance with some implementations of the present disclosure. As shown in fig. 6, the outer contour of the region 132 includes contour points 610, 612, 614, and 616, and the outer contour of the region 510 includes contour points 614, 620, 622, and 624, where the two regions 132 and 520 collectively refer to 4+4=8 contour points. The union of the two regions may be determined based on the geometric relationship of the two regions. The union includes contour points 610, 612, 620, 622, 624, and 616, and the number of contour points of the union is 6. Therefore, the exclusion area is represented based on the union, so that the number of contour points related to the exclusion area can be reduced, and further the cost of subsequent computing resources related to polygon processing is reduced.

FIG. 6 is merely a case where a union is determined based on two identified regions, and according to one exemplary implementation of the present disclosure, the number of identified regions will continue to increase as the image is continually processed by the producer. In the case of a large number of identified regions, determining the exclusion region based on the union may greatly reduce the number of contour points involved in the exclusion region, thereby improving subsequent processing performance. Another example of determining a union is described with reference to fig. 7, which fig. 7 illustrates a block diagram 700 of a process for determining a union of a plurality of identified regions in accordance with some implementations of the present disclosure.

Fig. 7 shows a case where there are 3 identified zones (zones 132, 510, and 730), when the outer contour of zone 132 includes contour points 610, 612, 614, and 616, the outer contour of zone 510 includes contour points 614, 620, 622, and 624, and zone 730 includes contour points 710, 712, 714, 716, 718, 614, 624, 720. At this time, the 3 regions include 16 contour points in total. Based on the geometric relationship between the 3 regions, the union of the 3 regions includes contour points 710, 712, 714, 716, 718, 620, 622, 720, and the number of contour points is 8. In this way, the number of contour points of the exclusion area can be greatly reduced, and the calculation efficiency of subsequent processing is improved.

According to one exemplary implementation of the present disclosure, after the target area of the target object currently being processed has been updated based on the identified area, the updated target area may be set to the type of the target object, returning to fig. 5 for further details. As shown in FIG. 5, the area associated with the wall may be represented by a polygon 530, and further the area shown in the legend 160 in the image may be identified as a wall. At this point, a region that would include 3 objects within the outer contour of region 210: a wall area shown with legend 160, a television area shown with legend 164, and a sound area shown with legend 540. In this way, a one-to-one correspondence relationship can be established between the regions and the objects, thereby facilitating reduction of complexity of subsequent processing.

According to one exemplary implementation of the present disclosure, the updated target area may be treated as a new identified area. At this point, the union of the new identified region and the previously identified region may be determined as an exclusion region for processing subsequent regions identified for other objects. By recording the union of the previously identified regions, each time a new region of the object is identified, only the difference set operation between the newly added region and the previous union is required. Specifically, a union U of the identified regions may be defined, if region a is newly added, a = a-U is performed, and U is further updated, i.e., U = U £ a; if the area B is newly added, B = B-U is executed, and U is further updated, namely U = U { (U {); if the area C is newly added, C = C-U is performed and U is further updated, i.e., U = U £ C. At this time, the number of the contour points for describing the union set is obviously smaller than the sum of the number of the contour points of the plurality of areas, so that the precision problem and the performance problem caused by a plurality of difference set operations can be greatly reduced. According to one exemplary implementation of the present disclosure, to ensure computational accuracy, an accuracy of 5 bits (or other number) after the decimal point may be preserved.

According to an exemplary implementation of the present disclosure, the above-described technical solution for identifying images may be invoked in a panoramic roaming production application. Here, the images may include a virtual roaming scene panorama image and/or a conventional captured image for rendering a real scene in a panoramic roaming production application. It will be appreciated that the panoramic image includes all-round information of the real scene, and thus the panoramic image has a larger field of view. Identifying objects in a panoramic image helps to improve the efficiency of identification, and individual peripheral objects related to the acquisition position of the panoramic image can be collectively processed in a single image.

According to an exemplary implementation of the present disclosure, the image to be processed may include an acquired image used to generate a panoramic image, e.g., a top view, a bottom view, a front view, a back view, a left view, a right view. Although the field of view of the captured image is not large enough, the object in the captured image is deformed to a lesser extent. Compared with the method that a plurality of line segments are needed to select the object with larger deformation in the panoramic image, the method that the region of the object is directly marked in the collected image enables a producer to easily specify the region of the object in a straight line mode, and therefore marking efficiency is improved.

According to one exemplary implementation of the present disclosure, contour points selected by a producer may be located on different planes in a real scene, and in order to improve the accuracy of identifying an object, the positions of the respective contour points from a target area specified by an action of the producer may be further detected. More details of determining the location of the contour points are described below with reference to fig. 8. Fig. 8 illustrates a block diagram 800 for determining whether multiple contour points of a target region lie in the same plane, according to some implementations of the present disclosure.

As shown in fig. 8, assuming that the producer designates the target area 830, the target area 830 spans the walls 120 and 820 in the real scene based on the coordinate mapping of the panoramic image. Since the walls 120 and 820 are located on different planes in the real scene, respectively, it can be considered that an error may occur in designating the target area 830, and thus an error message can be provided to prompt the producer to re-designate the target area.

According to one exemplary implementation of the present disclosure, assuming that a plurality of contour points of the target region are located on the same plane in the real scene, it may be considered that the target region is located on the same plane in the real scene at this time, and thus the identified region coincides with the object position of the real scene. At this time, the target area of the target object may be updated based on the above-described procedure. In this way, the accuracy of image processing can be improved to identify the region in which the object is located in the image in a more accurate manner.

According to an exemplary implementation of the present disclosure, if it is determined that the at least one identified region and the plurality of outline points are located in the same plane, a type of each identified object may be further determined in order to determine whether a potential region merging process may be performed. The identified region and the target region of the identified object may be merged if it is determined that the type of the identified object in the at least one identified object is the same as the type of the target object. Returning to FIG. 5 for more details regarding merging, assuming that region 510 in FIG. 5 was previously identified as a wall, i.e., the same type as the current region, the producer may be prompted to: both areas are walls and the producer is asked whether to perform a merge operation.

Assuming that region 510 is the region previously identified as being faulty, the fabricator may instruct the merge operation to be performed. In this way, the region 510 may be merged to the current region, and the polygon of the current region may be updated to:

polygon = { outer contour: [ (x) ₁₁ ,y ₁₁ ),(x ₁₂ ,y ₁₂ ),…,(x _1n1 ,y _1n1 )]And excluding the area: [ (x) ₂₁ ,y ₂₁ ),(x ₂₂ ,y ₂₂ ),…,(x _2n2 ,y _2n2 )]}

In this way, potential errors of previous identification processes may be verified based on the relative type of the identified region, thereby improving the accuracy of the identification process. According to one exemplary implementation of the present disclosure, if it is determined that the type of each identified object is different from the type of the target object, the merging process may be deemed to be unable to be performed. At this time, the target area of the target object may be updated based on the above-described technical solution. According to an exemplary implementation of the present disclosure, the above-described technical solution may be invoked only when needed. In this way, the overhead of updating the target area of the current identification operation based on the identified area if unnecessary may be reduced.

With example implementations of the present disclosure, the target area of the current target object specified by the identifying action may be updated based on the respective identified areas. In this way, based on the difference set between the target region and the identified regions, the target object may be represented in a single region without performing region splitting. Therefore, a one-to-one mapping relation can be established between the regions and the objects, and complexity of a later process of generating the panoramic roaming is simplified.

Example procedure

Fig. 9 illustrates a flow diagram of a method 900 for identifying regions of objects in an image, according to some implementations of the present disclosure. Specifically, at block 910, at least one identified region of at least one identified object in the image is obtained. At block 920, an action is detected for specifying a target region of a target object in an image. At block 930, the target area of the target object specified by the action is updated based on the at least one identified area.

According to one exemplary implementation of the present disclosure, updating the target area includes: in response to determining that the at least one identified region is located within the target region, an updated target region is determined based on a difference between the target region and the at least one identified region.

According to one exemplary implementation of the present disclosure, determining an updated target area includes: determining a union of at least one identified region; and determining an updated target region based on the difference between the target region and the union.

According to an exemplary implementation of the present disclosure, the method 900 further includes: the updated target region is represented by polygon data comprising a sequence of contour points of the target region and a sequence of contour points of the union.

According to an exemplary implementation of the present disclosure, the method 900 further includes: setting the updated target area as the type of the target object; and adding the target object to the at least one identified object.

According to an example implementation of the present disclosure, the method 900 is invoked in a panoramic roaming production application, and the image includes at least any one of the following virtual roaming scenes for rendering a real scene in the panoramic roaming production application: panoramic images and images.

According to an exemplary implementation of the present disclosure, the method 900 further includes: determining whether a plurality of contour points of a target area are located on the same plane in a real scene; and in response to determining that the plurality of contour points lie in the same plane, updating a target region of the target object.

According to one exemplary implementation of the present disclosure, updating the target area of the target object includes: determining at least one type of at least one identified object; and in response to determining that the type of the identified object of the at least one identified object is the same as the type of the target object, merging the identified region and the target region of the identified object.

According to an exemplary implementation of the present disclosure, the method 900 further includes: in response to determining that all of the at least one type is different from the type of the target object, updating the target area of the target object.

Example apparatus and devices

Fig. 10 illustrates a block diagram of an apparatus 1000 for identifying a region of an object in an image, in accordance with some implementations of the present disclosure. The apparatus 1000 comprises: an acquisition module 1010 configured to acquire at least one identified region of at least one identified object in an image; a detection module 1020 configured to detect an action for specifying a target region of a target object in an image; and an update module 1030 configured for updating the target area of the target object specified by the action based on the at least one identified area.

According to an exemplary implementation of the present disclosure, the update module 1030 includes: a region determination module configured to determine an updated target region based on a difference between the target region and the at least one identified region in response to determining that the at least one identified region is located within the target region.

According to one exemplary implementation of the present disclosure, the region determining module includes: a union determination module configured to determine a union of at least one identified region; and a difference-based region determination module configured to determine an updated target region based on a difference between the target region and the union.

According to an exemplary implementation of the present disclosure, the apparatus 1000 further includes: a representation module configured to represent the updated target region with polygon data, the polygon data comprising a sequence of contour points of the target region and a sequence of contour points of the union.

According to an exemplary implementation of the present disclosure, the apparatus 1000 further includes: setting a model to set the updated target area as the type of the target object; and an adding module configured to add the target object to the at least one identified object.

According to an example implementation of the present disclosure, the apparatus 1000 is invoked in a panoramic roaming production application, and the image comprises at least any one of the following virtual roaming scenes for rendering a real scene in the panoramic roaming production application: panoramic images and images.

According to an exemplary implementation of the present disclosure, the apparatus 1000 further includes: a plane determination module configured to determine whether a plurality of contour points of a target area are located on the same plane in a real scene; and the update module is further configured for updating the target area of the target object in response to determining that the plurality of contour points lie in the same plane.

According to an exemplary implementation of the present disclosure, the update module 1030 further includes: a type determination module configured to determine at least one type of the at least one identified object; and a merging module configured for merging the identified region and the target region of the identified object in response to determining that the type of the identified object of the at least one identified object is the same as the type of the target object.

According to an example implementation of the present disclosure, the update module 1030 is further configured to: in response to determining that all of the at least one type is different from the type of the target object, updating the target area of the target object.

Fig. 11 illustrates a block diagram of a device 1100 capable of implementing multiple implementations of the present disclosure. It should be understood that the computing device 1100 illustrated in FIG. 11 is merely exemplary and should not constitute any limitation as to the functionality or scope of the implementations described herein. The computing device 1100 shown in fig. 11 may be used to implement the methods described above.

As shown in fig. 11, computing device 1100 is in the form of a general purpose computing device. Components of computing device 1100 may include, but are not limited to, one or more processors or processing units 1110, memory 1120, storage device 1130, one or more communication units 1140, one or more input devices 1150, and one or more output devices 1160. The processing unit 1110 may be a real or virtual processor and can perform various processes according to programs stored in the memory 1120. In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of computing device 1100.

Computing device 1100 typically includes a number of computer storage media. Such media may be any available media that is accessible by computing device 1100 and includes, but is not limited to, volatile and non-volatile media, removable and non-removable media. The memory 1120 may be volatile memory (e.g., registers, cache, random Access Memory (RAM)), non-volatile memory (e.g., read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory), or some combination thereof. Storage device 1130 may be a removable or non-removable medium and may include a machine-readable medium, such as a flash drive, a magnetic disk, or any other medium that may be capable of being used to store information and/or data (e.g., training data for training) and that may be accessed within computing device 1100.

The computing device 1100 may further include additional removable/non-removable, volatile/nonvolatile storage media. Although not shown in FIG. 11, a magnetic disk drive for reading from and writing to a removable, non-volatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to a bus (not shown) by one or more data media interfaces. The memory 1120 may include a computer program product 1125 having one or more program modules configured to perform the various methods or acts of the various implementations of the disclosure.

The communication unit 1140 enables communication with other computing devices over a communication medium. Additionally, the functionality of the components of computing device 1100 may be implemented in a single computing cluster or multiple computing machines, which are capable of communicating over a communications connection. Thus, the computing device 1100 may operate in a networked environment using logical connections to one or more other servers, network Personal Computers (PCs), or another network node.

The input device 1150 may be one or more input devices such as a mouse, keyboard, trackball, or the like. Output device(s) 1160 may be one or more output devices such as a display, speakers, printer, etc. Computing device 1100 can also communicate with one or more external devices (not shown), such as storage devices, display devices, etc., communication devices with one or more devices that enable a user to interact with computing device 1100, or communication with any devices (e.g., network cards, modems, etc.) that enable computing device 1100 to communicate with one or more other computing devices, as desired, via communication unit 1140. Such communication may be performed via input/output (I/O) interfaces (not shown).

According to an exemplary implementation of the present disclosure, a computer-readable storage medium having stored thereon computer-executable instructions is provided, wherein the computer-executable instructions are executed by a processor to implement the above-described method. According to an exemplary implementation of the present disclosure, there is also provided a computer program product, tangibly stored on a non-transitory computer-readable medium and comprising computer-executable instructions that are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is provided, on which a computer program is stored, which when executed by a processor implements the method described above.

Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus, devices and computer program products implemented in accordance with the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processing unit of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various implementations of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The foregoing has described implementations of the present disclosure, and the above description is illustrative, not exhaustive, and not limited to the implementations disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The terminology used herein was chosen in order to best explain the principles of various implementations, the practical application, or improvements to the technology in the marketplace, or to enable others of ordinary skill in the art to understand various implementations disclosed herein.

Claims (20)

1. A method for identifying a region of an object in an image, comprising:

obtaining at least one identified region of at least one identified object in the image;

detecting an action for specifying a target region of a target object in the image; and

updating the target area of the target object specified by the action based on the at least one identified area.

2. The method of claim 1, wherein updating the target area comprises: in response to determining that the at least one identified region is located within the target region, determining an updated target region based on a difference between the target region and the at least one identified region.

3. The method of claim 2, wherein determining the updated target region comprises:

determining a union of the at least one identified region; and

determining the updated target region based on a difference between the target region and the union.

4. The method of claim 3, further comprising: representing the updated target region using polygon data comprising a sequence of contour points of the target region and a sequence of contour points of the union.

5. The method of claim 1, further comprising:

setting the updated target area as the type of the target object; and

adding the target object to the at least one identified object.

6. The method of claim 5, wherein the method is invoked in a panoramic roaming production application, and the image comprises at least any one of the following of a virtual roaming scene for rendering a real scene in the panoramic roaming production application: panoramic images and images.

7. The method of claim 6, further comprising:

determining whether a plurality of contour points of the target region are located on the same plane in the real scene; and

updating the target region of the target object in response to determining that the plurality of contour points lie in the same plane.

8. The method of claim 7, wherein updating the target region of the target object comprises:

determining at least one type of the at least one identified object; and

merging the identified region of the identified object and the target region in response to determining that a type of an identified object of the at least one identified object is the same as the type of the target object.

9. The method of claim 8, further comprising: in response to determining that all of the at least one type is different from the type of the target object, updating the target region of the target object.

10. An apparatus for identifying a region of an object in an image, comprising:

an acquisition module configured to acquire at least one identified region of at least one identified object in the image;

a detection module configured to detect an action for specifying a target region of a target object in the image; and

an update module configured to update the target area of the target object specified by the action based on the at least one identified area.

11. The apparatus of claim 10, wherein the update module comprises: a region determination module configured to determine an updated target region based on a difference between the target region and the at least one identified region in response to determining that the at least one identified region is located within the target region.

12. The apparatus of claim 11, wherein the region determination module comprises:

a union determination module configured to determine a union of the at least one identified region; and

a difference-based region determination module configured to determine the updated target region based on a difference between the target region and the union.

13. The apparatus of claim 12, further comprising: a representation module configured to represent the updated target region with polygon data comprising a sequence of contour points of the target region and a sequence of contour points of the union.

14. The apparatus of claim 10, further comprising:

setting a fumble to set the updated target area as the type of the target object; and

an adding module configured to add the target object to the at least one identified object.

15. The apparatus of claim 14, wherein the apparatus is invoked in a panoramic roaming production application, and the image comprises at least any one of the following of a virtual roaming scene for rendering a real scene in the panoramic roaming production application: panoramic images and images.

16. The apparatus of claim 15, further comprising:

a plane determination module configured to determine whether a plurality of contour points of the target region lie in the same plane in the real scene; and

the update module is further configured to update the target region of the target object in response to determining that the plurality of contour points lie in the same plane.

17. The apparatus of claim 16, wherein the update module further comprises:

a type determination module configured to determine at least one type of the at least one identified object; and

a merging module configured to merge an identified region of the identified object and the target region in response to determining that a type of an identified object of the at least one identified object is the same as the type of the target object.

18. The apparatus of claim 17, wherein the update module is further configured to: in response to determining that all of the at least one type is different from the type of the target object, updating the target region of the target object.

19. An electronic device, comprising:

at least one processing unit; and

at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions when executed by the at least one processing unit causing the electronic device to perform the method of any of claims 1-9.

20. A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, causes the processor to carry out the method according to any one of claims 1 to 9.

Images