US20090201311A1

US20090201311A1 - Electronic manifest of underground facility locate marks

Info

Publication number: US20090201311A1
Application number: US12/363,009
Authority: US
Inventors: Steven Nielsen; Curtis Chambers
Original assignee: CertusView Techonologies LLC
Current assignee: CertusView Techonologies LLC
Priority date: 2008-02-12
Filing date: 2009-01-30
Publication date: 2009-08-13
Also published as: US20090202111A1; US8265344B2; US9471835B2; US8340359B2; US20130315449A1; US20090202101A1; US20140321717A1; US8532342B2; US20170140562A1; US20090202110A1; US20090204625A1; US8416995B2

Abstract

Methods and apparatus for generating a searchable electronic record of a locate operation in which one or more physical locate marks are applied by a technician to indentify a presence or an absence of at least one underground facility within a dig area. A digital image of a geographic area comprising the dig area is electronically received, and at least a portion of the received digital image is displayed on a display device. One or more digital representations of the physical locate mark(s) applied by the locate technician are added to the displayed digital image so as to generate a marked-up digital image. Information relating to the marked-up digital image is electronically transmitted and/or electronically stored so as to generate the searchable electronic record of the locate operation.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit under 35 U.S.C. §120 as a continuation of U.S. patent application Ser. No. 12/029,732, filed Feb. 12, 2008, entitled “Electronic Manifest of Underground Facility Locate Marks.”

BACKGROUND

Excavators are required to notify underground facility owners/operators in advance of their excavation activities and to describe and communicate the geographic area of those activities to underground facility owners/operators. The geographic area so described is commonly referred to as “the dig area.” In turn, facility owners/operators are required to determine if they own or operate any underground facilities at an identified dig area. The presence of underground facilities at a dig area is generally detected using a device commonly referred to as a “locate wand.” Locate wands use a number of electronic methods to detect the presence of underground facilities. The location of those underground facilities, if any, which exist within a dig area, is marked using paint or some other physical marking system, such as flags. Paint is generally applied as a sequence of dashes or dots on the surface (grass, dirt, asphalt, concrete, etc.) directly above the underground facility and is color-coded to indicate to the excavator the type (e.g., gas, water, sewer, power, telephone, cable television, etc.) of the underground facility present. Flags, which also may identify the underground facility via color-coding, can be placed in the ground directly above the underground facility being marked. Paint and/or flags can be dispensed using various devices. The application of paint, flags, or some other marking object to indicate the presence of an underground facility is called a “locate.” The marks resulting from a locate are commonly called underground facility “locate marks.”
Underground facility owners/operators may perform locates with in-house employees or choose to hire independent contract locating firms to perform locates on their behalf. Generally, the person performing the locate operation is called a locate technician. The set of instructions necessary for a locate technician to perform a locate operation may be called a “ticket.” A ticket might specify, for example, the address or description of the dig area to be marked, the day and/or time that the dig area is to be marked, and/or whether the user is to mark the dig area for telecommunications (e.g., telephone and/or cable television), power, gas, water, sewer, or some other underground facility.
It is generally recommended, or in some jurisdictions required, to document the type and number of underground facilities located, i.e. telephone, power, gas, water, sewer, etc., and the approximate geographic location of the locate marks. Often times it is also recommended or required to document the distance, or “offset” of the locate marks from environmental landmarks that exist at the dig area. An environmental landmark may include any physical object that is likely to remain in a fixed location for an extended period of time. Examples of an environmental landmark may include a tree, a curb, a driveway, a utility pole, a fire hydrant, a storm drain, a pedestal, a water meter box, a manhole lid, a building structure (e.g., a residential or office building), or a light post. For example, a telephone cable located two and a half meters behind the curb of a residential street would be documented as being offset two and a half meters behind the curb. These offsets serve as evidence supporting the location of the locate marks after those locate marks may have been disturbed by the excavation process.
Documentation of some or all of the information regarding a locate operation is often called a “manifest.” A manifest may typically contain a variety of information related to a locate operation including a sketch or drawing of the dig area that identifies the approximate location of the locate marks and environmental landmarks present at the dig area; the time and date the locate operation was performed; identification of the entity and the locate technician performing the locate operation; the entity requesting the locate operation; the geographic address of the dig area; the type of markings used for the locate operation (e.g., colored paint, flags, or other markers); notes from the locate technician; and/or a technician signature.
If performing locate operations with in-house employees, each individual underground facility owner/operator generally documents on the manifest only the existence of its facilities and the approximate location of its locate marks. If an independent contract locating firm is hired to perform locates for more than one underground facility owner/operator, the contract locating firm may document on the manifest some or all of the underground facilities at the dig area that it located and the approximate location of all the locate marks.
Currently, locate marks are generally documented using a sketching process which results in the creation of a paper manifest. Sketches are produced by hand, are not to scale, prone to human error, and costly in drafting time spent by the locate technician. They are stored manually or in some jurisdictions are digitally scanned/photographed and the image stored electronically. Because the manifests are stored as paper or digital images, they are not easily interrogated for data in any mechanized way.

SUMMARY

According to one aspect, a method performed by a device may include receiving information regarding a particular geographic area; retrieving an aerial image of the particular geographic area; displaying the aerial image; determining an approximate geographic location of a locate mark denoting an underground facility; overlaying, on the displayed aerial image, information concerning the geographic location of the locate mark denoting the underground facility; and storing the aerial image and the information concerning the approximate geographic location of the locate mark denoting the underground facility.
According to another aspect, a device may include a memory to store aerial images of a plurality of geographic areas and a processing unit. The processing unit may receive information regarding a particular one of the geographic areas; retrieve one of the aerial images from the memory based on the received information; receive information concerning an approximate geographic location of an underground facility located within the particular geographic area; present, on the retrieved aerial image, information concerning the approximate geographic location of the underground facility locate marks; and store the retrieved aerial image and the information concerning the approximate geographic location of the underground facility locate mark.
According to a further aspect, a system may include means for identifying a geographic area; means for retrieving an aerial image of the geographic area; means for displaying the retrieved image; means for receiving input from a user concerning an approximate geographic location of underground facility locate marks that are located within the geographic area; means for presenting, on the displayed aerial image, a marking that identifies the approximate geographic location of the underground facility locate marks within the geographic area; and means for storing the displayed aerial image with the marking that identifies the approximate geographic location of the underground facility locate marks.
According to another aspect, a system may include a central server to store a plurality of aerial images of a corresponding plurality of geographical locations; and a user device. The user device may identify a particular geographic location; determine whether the user device stores an aerial image corresponding to the particular geographic location; retrieve the aerial image from the central server when the user device does not store the aerial image corresponding to the particular geographic location; retrieve the aerial image from a local memory when the user device stores the aerial image corresponding to the particular geographic location; display the aerial image; receive information regarding an approximate location of underground facility locate marks located at the particular geographic location; and present, on the displayed aerial image, a marking that identifies the approximate location of the underground facility locate marks.
According to still another aspect, a method performed by a device may include receiving information regarding a particular geographic area; retrieving an aerial image of the particular geographic area; receiving, from a GPS-enabled device, information regarding an approximate location of underground facility locate marks; displaying, as a combined image, the aerial image and the information regarding the approximate location of underground facility locate marks; and storing the combined image.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate one or more embodiments described herein and, together with the description, explain these embodiments. In the drawings:

FIG. 1 is a diagram of a concept described herein;

FIG. 2 is a diagram of an exemplary network in which systems and methods described herein may be implemented;

FIG. 3 is a diagram of exemplary components of the user device of FIG. 2;

FIG. 4 is a diagram of exemplary components of the central server of FIG. 2;

FIG. 5 is a diagram of exemplary software routines for components of FIG. 2;

FIG. 6 is a flowchart of an exemplary process for creating an electronic manifest of underground facility locate marks;

FIG. 7 is a diagram of an exemplary data set that may be stored in the memory of FIG. 3 and/or FIG. 4; and

FIG. 8 is a diagram of an exemplary user interface that may be presented via the user device of FIG. 2.

DETAILED DESCRIPTION

The following detailed description refers to the accompanying drawings. The same reference numbers in different drawings may identify the same or similar elements. Also, the following detailed description does not limit the invention.

Overview

FIG. 1 is a diagram of a concept described herein. When locating underground facilities at a geographic location, such as at a dig area 100 associated with a residence or a business, it may be beneficial to document locate marks in a permanent and reproducible manner. For example, a locate technician may locate and mark underground facilities using a locating device and/or a marking device. A locating device may generally be defined as a locating wand or another device used to detect the presence of underground facilities; while a marking device may generally be defined as any tool to apply paint or other material to a surface, such as a paint wand. The locate technician may use paint, flags, or some other object with a particular color or other characteristic to mark the location of an underground facility. Referring to the example shown in FIG. 1, the locate technician may use red paint to mark underground power lines 110, orange paint to mark telecommunications (e.g., telephone and/or cable television) lines 120, and yellow paint to mark gas lines 130.
The locate technician may also identify one or more environmental landmarks that are present at or near the dig area and/or determine the distance between the environmental landmark(s) and the located underground facility. For example, a transformer 140 may be indicated as an environmental landmark, as shown in FIG. 1. The geographic location of transformer 140 may be used to measure offsets to other locate marks in the dig area.
As described herein, documentation of some or all of this information regarding a locate operation is created as an electronic manifest. An electronic manifest, as used herein, may generally refer to one or more computer-readable files that include some or all of the information in a manifest. The electronic manifest may be created using aerial images of a dig area, such as dig area 100, that may be combined with information that is added by the user about the locate marks and/or the environmental landmarks. In other implementations, an electronic manifest may be created using aerial images of a dig area combined with information about locate marks that is provided by other sources. Other implementations may use aerial images of a dig area combined with information that is added by the user and information that is provided by other sources. As used herein, an “aerial image” is intended to be broadly interpreted as any image taken from above the earth's surface, such as, for example, images generated using a satellite, airplane, helicopter, or other moving or fixed device. Also as used herein, a “user” may refer to any person operating a device to create an electronic manifest, such as a locate technician, a site supervisor, or any other person or group of people.

Exemplary Network

FIG. 2 is a diagram of an exemplary network 200 in which systems and methods described herein may be implemented. As shown in FIG. 2, the network 200 may include a user device 210 connected to a central server 220 and an image server 230 via a network 240. A single user device 210, central server 220, and image server 230 have been illustrated as connected to network 240 for simplicity. In practice, there may be more or fewer user devices and/or servers. For example, in one alternative implementation, the user device 210 may operate as a comprehensive device and, thus, the network 200 may include no central server, with user device 210 communicating directly through network 240 to image server 230. Also, in some instances, the user device 210 may perform one or more of the functions of the central server 220 and/or central server 220 may perform one or more of the functions of the user device 210. In still another implementation, multiple user devices 210 may be connected to the central server 220 through the network 240.
The user device 210 may encompass a computer device, such as a laptop computer, a small personal computer, a tablet device, a personal digital assistant (PDA), a mobile computing device, a touch-screen device, or generally any device including or connecting to a processor and a display. The user device 210 may be portable so as to be separately carried by the user performing a locate operation. Alternatively, the user device 210 may be integrated with or affixed to another moveable object, such as a vehicle.
The central server 220 may include a computer device that may store information received from or provided to the user device 210 and/or the image server 230. The central server 220 may include storage capacity and/or optionally include networked access to one or more separate hardware components, such as images cache 235, to store cached images and the like.
The image server 230 may include a computer device that may store and provide aerial images of geographic locations The image server 230 may be associated with the same, or a different, party that maintains the central server 220. For example, the image server 230 may be associated with a party that provides aerial images for a fee.
The network 240 may include a local area network (LAN), a wide area network (WAN), a telephone network, such as the Public Switched Telephone Network (PSTN) or a cellular network, an intranet, the Internet, a communications link, or a combination of networks. The user device 210, central server 220, and image server 230 may connect to the network 240 via wired and/or wireless connections. The user device 210 and central server 220 may communicate using any communication protocol.

Exemplary User Device Architecture

FIG. 3 is a diagram of exemplary components of the user device 210. The user device 210 may include a bus 310, a processing unit 320, a memory 330, an input device 340, an output device 350, a location identification unit 360, and a communication interface 370. In another implementation, the user device 210 may include more, fewer, or different components. For example, the location identification unit 360 may not be included, or the location identification unit 360 may be included as a device located external to the user device 210, such as a device worn or carried by a user of the user device 210.
The bus 310 may include a path that permits communication among the components of the user device 210. The processing unit 320 may include a processor, a microprocessor, or processing logic that may interpret and execute instructions. The memory 330 may include a random access memory (RAM), a read only memory (ROM), a memory card, a magnetic and/or optical recording medium and its corresponding drive, or another type of memory device. Generally, the memory 330 may be sufficient to store and manipulate aerial images, such as those stored in a local image cache 335. In one implementation, the local image cache 335 may include one or more aerial images of a dig area to be marked by a user. In another implementation, the local image cache 335 may include a series of aerial images that correspond to the geographical region to which a particular user is assigned. For example, local image cache 335 may include a collection of high-resolution images of a particular zip code or town. In still another implementation, the local image cache 335 may include an entire set of aerial images intended to be made available to multiple users.
The input device 340 may include one or more mechanisms that permit a user to input information to the user device 210, such as a keyboard, a keypad, a touchpad, a mouse, a stylus, a touch screen, a camera, or the like. Alternatively, or additionally, the input device 340 may include a microphone that can capture a user's intent by capturing the user's audible commands. Alternatively, or additionally, the input device 340 may interact with a device that monitors a condition of the user, such as eye movement, brain activity, or heart rate. The output device 350 may include a mechanism that outputs information to the user, such as a display, a speaker, or the like.
The location identification unit 360 may include a device that can determine its geographic location to a certain degree of accuracy, such as a global positioning system (GPS) or a global navigation satellite system (GNSS) receiver. In another implementation, the location identification unit 360 may include a device that determines location using another technique, such as tower (e.g., cellular tower) triangularization. The location identification unit 360 may receive location tracking signals (e.g., GPS signals) and determine its location based on these signals. In one implementation, location identification unit 360 may be capable of determining its location within approximately thirty centimeters or less.
The communication interface 370 may include any transceiver-like mechanism that enables user device 210 to communicate with other devices and/or systems. For example, the communication interface 370 may include mechanisms for communicating with another device or system via a network. For example, the communication interface 370 may enable communications between the user device 210 and the central server 220 and/or image server 230 over network 240.
As will be described in detail below, user device 210 may perform certain operations relating to the documentation of locate operations and/or the creation of an electronic manifest User device 210 may perform these operations in response to the processing unit 320 executing software instructions contained in a computer-readable medium, such as the memory 330. A computer-readable medium may be defined as a physical or logical memory device.
The software instructions may be read into the memory 330 from another computer-readable medium, or from another device via the communication interface 370. The software instructions contained in the memory 330 may cause processing unit 320 to perform processes that will be described later. Alternatively, hardwired circuitry may be used in place of, or in combination with, software instructions to implement processes described herein. Thus, implementations described herein are not limited to any specific combination of hardware circuitry and software.

Exemplary Central Server Architecture

FIG. 4 is a diagram of exemplary components of the central server 220. The central server 220 may include a bus 410, a processing unit 420, a memory 430, and a communication interface 440. In another implementation, the central server 220 may include more, fewer, or different components. For example, an input device and/or an output device (not shown) may be included, as necessary.
The bus 410 may include a path that permits communication among the components of the central server 220. The processing unit 420 may include a processor, a microprocessor, or processing logic that may interpret and execute instructions. The memory 430 may include a magnetic and/or optical recording medium and its corresponding drive, a RAM, a ROM, a memory card, or another type of memory device suitable for high capacity data storage. Generally, the memory 430 may be sufficient to store aerial images of particular geographic locations, such as those stored in a central image cache 435. In one implementation, the central image cache 435 may include a set of aerial images that correspond to the geographical regions to which a group of users are assigned. In still another implementation, the central image cache 435 may include the entire set of aerial images intended to be made available to any of a group of users. For example, central image cache 435 may include a collection of high-resolution aerial images of a particular county, state or other geographic region. In another implementation, as shown in FIG. 2, central image cache 435 may be replaced or supplemented with one or more networked storage components, such as image cache 235.
The communication interface 440 may include any transceiver-like mechanism that enables the central server 220 to communicate with other devices and/or systems. For example, the communication interface 440 may include mechanisms for communicating with another device or system via a network. For example, the communication interface 440 may enable communications between the central server 220 and the user device 210 and/or image server 230 over network 240.
As will be described in detail below, the central server 220 may perform certain operations to facilitate the documentation of locate operations and/or the creation of an electronic manifest. The central server 220 may perform these operations in response to the processing unit 420 executing software instructions contained in a computer-readable medium, such as the memory 430.
The software instructions may be read into the memory 430 from another computer-readable medium, or from another device via the communication interface 440. The software instructions contained in the memory 430 may cause processing unit 420 to perform processes that will be described later. Alternatively, hardwired circuitry may be used in place of or in combination with software instructions to implement processes described herein. Thus, implementations described herein are not limited to any specific combination of hardware circuitry and software.

Exemplary Routines

FIG. 5 is a diagram of exemplary software routines for the components shown in FIG. 2. The central server 220 may include an image retrieval routine 510 and a central image cache routine 520. The user device 210 may include a synchronize routine 530, a local image cache routine 540, an image display routine 550, a user input routine 560, and a ticket manager routine 570. As discussed in more detail herein, the examples of routines associated with the central server 220 and the user device 210 may be interchangeable between each hardware component. Furthermore, some or all of routines 510, 520, 530, 540, 550, 560, and 570 need not be performed exclusively by any one hardware component.
Still referring to FIG. 5, the image server 230 may store a library of aerial images. Generally, the aerial images may be of sufficient resolution at an optimal elevation to be useful as a record of the locate operation. The aerial images from the image server 230 may include geocoding or other geographical identification metadata and may be provided in any computer-readable format, such as JPEG file interchange format (JPEG), tagged image file format (TIFF), portable document format (PDF), graphics interchange format (GIF), bitmap (BMP), portable network graphics (PNG), Windows® metafile (WMF), and/or the like. Also, aerial images from the image server 230 may include a combination of images or overlays, such as overlays of street names, regions, landmark descriptions, and/or other information about areas displayed in an image. The aerial images from the image server 230 may be supplied by a third-party provider if the coverage area of the third-party image provider overlaps with the desired area of the user.
The central image cache routine 510 and the image retrieval routine 520 of the central server 220 may include a variety of functionalities. In certain implementations, the central image cache routine 510 may receive information about specific tickets and parse tickets in order to discern location information. For example, a ticket may identify the dig area by an address of the property or by geographic coordinates. The ticket might specify, for example, the address or description of the dig area to be marked, the day and/or time that the dig area is to be marked, and/or whether the user is to mark the dig area for telecommunications (e.g., telephone and/or cable television), power, gas, water, sewer, or some other underground facility.
The central image cache routine 510 may also convert dig area location information to latitude/longitude coordinates or other coordinates. When location information from a ticket is sufficiently precise to allow for identification of corresponding imagery, the central image cache routine 510 may calculate the image extent (which may be generally defined as the bounding region of the dig area of interest), and update the ticket with the calculated extent. In one implementation, the central image cache routine 510 may determine image date, coordinates, and resolution of each image that may be stored in the central image cache 435 or in another location. In another implementation, when location information from a ticket is imprecise (or “fuzzy”), the central image cache routine 510 may mark the ticket to indicate that no corresponding image was able to be retrieved based on the ticket information.
In another implementation, central image cache 510 may identify an image to retrieve based on GPS coordinates of a GPS-enabled device associated with a user. For example, a user may arrive at an excavation site in a GPS-enabled vehicle and the GPS information from the vehicle may be used to identify coordinates corresponding to an image to be retrieved. GPS coordinates may also be obtained from other GPS-enabled devices being used by or in the vicinity of the user. As used herein a GPS-enabled device may include any device or combination of devices capable of interfacing with a global navigation satellite system, geo-spatial positioning system, or other location-identification system to determine a location. Examples of GPS-enabled devices may include a marking device (e.g., a paint wand) with an integrated GPS receiver; a locating device (e.g., a locating wand) with a GPS receiver; a wearable GPS-enabled device; a vehicle-mounted GPS system; certain PDAs, computers, and cellular telephones; and stand-alone GPS-enabled systems.
In still another implementation, central image cache 510 may identify one or more images to request based on a designated geographical area assigned to a user. For example, a user may be assigned to work in several dig areas associated with a particular section of a neighborhood. The user may input coordinates associated with the entire selected section of the neighborhood, and central image cache 510 may then retrieve images for those coordinates.
The image retrieval routine 520 catalogues and stores images from the image server 230 to the central server 220. For example, images may be stored in the central image cache 435 in the memory 430 of the central server 220. In one implementation, the image retrieval routine 520 may query the central image cache 435 or other cache for an image associated with a particular dig area relating to a ticket of interest, and determine, based on (for example) the age and resolution of the cached image, whether the image in the central image cache 435 needs to be updated from the image server 230.
In another implementation, the image retrieval routine 520 may interface with multiple image providers and image servers 230. The image retrieval routine 520 may determine which image provider is the best source for the image corresponding to a particular dig area relating to a ticket of interest based on algorithms that factor, for example, each image provider's geographical coverage, image resolution, cost, and availability. Regarding geographical coverage, it will be beneficial to confirm that the image provider's area of coverage includes the desired extent (in other words, the entire geographical region of interest to the user).
Regarding image resolution, available resolution may be measured in meters (or centimeters, feet, or inches) per pixel. For example, one provider may offer thirty centimeters per pixel, while another offers fifteen centimeters or less per pixel, for the same coverage area. If an image is requested at a standard altitude, then the image retrieval routine 520 may choose a pre-defined optimal scale (for example, thirty centimeters per pixel for a rural area, but fifteen centimeters per pixel for an urban area) and determine which provider provides images at the pre-defined optimal scale. Alternatively, if the image of interest is at a less granular scale (for example, a community or neighborhood image that allows the locator to pan around the image), then resolution may not be a significant factor.
Regarding cost, the image retrieval routine 520 may have access to pricing information for a variety of image providers. The image retrieval routine 520 may identify which provider has the lowest cost for the desired image. Cost analysis may be based on images desired for an individual ticket or the algorithm may account for a group of image requests, including volume incentives and/or penalties from each image provider
Regarding availability of image providers, the image retrieval routine 520 may identify what providers are available and/or operational. Also, if an image provider has a regular latency profile (for example, if a provider has a particular server that is busiest 3-5 PM Pacific time), then the image retrieval routine 520 may manage requests to be provided to another image provider or to a particular server of that image provider to efficiently load share the image retrieval.
When an image provider is selected, the image retrieval routine 520 may download the image from the selected image provider's server, which may be an image server 230. The downloaded image may be stored locally, for example, in the central image cache 435.
It should be understood that some of the routines and/or functionalities described above with respect to the central image cache routine 510 and the image retrieval routine 520 may be performed by one or both of the routines 510 and 520 above, and the arrangement of functionalities are not limited to the implementations disclosed herein.
The synchronize routine 530 for user device 210 may ensure that images already stored and manipulated on the user device 210 correspond to images stored in the central server 220. When a user performing a locate operation identifies a ticket or dig area, the synchronize routine 530 may check if an image exists in the central server 220 that matches the extent requested, and if the matching image is up-to-date in, for example, the local image cache 335. The synchronize routine 530 may also synchronize images from the central server 220 cache and store copies locally in the user device 210.
If the ticket has a valid extent (i.e., a recognizable boundary), the local image cache routine 540 may associate the ticket information with an image matching the extent. The local image cache routine 540 may load the image from the local image cache 335. If the ticket does not have a valid extent, the local image cache routine 540 may accept address information that is entered by the user. Alternatively, the local image cache routine 540 may read the local address information from the ticket or from a GPS-enabled device in communication with the user device 210 so that address information may be pre-entered for the user to the extent possible. Address information may include, for example, a street address, street name, city, state and/or zip code. If either none or multiple stored addresses appear to be associated with particular address information, the local image cache routine 540 may display a list of best match addresses from which a user can select.
Once an image is loaded from the local cache 335, image display routine 550 may provide a variety of view options for the user. For example, the image display routine 550 may support zooming in and out of the image by changing the image scale. Also, the image display routine 550 may support panning horizontally and vertically in the image. Furthermore, the image display routine 550 may support “roaming” outside the boundaries of the initial extent. Roaming generally occurs when the user zooms or pans, such that images beyond the boundaries of the stored images may be required to be retrieved (using, for example, synchronize routine 530) from either the local image cache 335 or the central server 220. The additional images retrieved from either the local image cache 335 or the central server 220 may be displayed and stitched together to display a complete image.
The user input routine 560 allows the user to add information to the image to create an electronic manifest. The user input routine 560 may accept user input from, for example, input device 340, and may support the addition of lines, freehand forms (or scribbling), shapes such as circles and rectangles, shading, or other markings which denote the approximate location of underground facilities which are present within the dig area. A drawing shape may generally be any kind of drawing shape or mark. The user input routine 560 may further enable drawing of underground facility locate marks for telecommunications (e.g., telephone and cable television), gas, power, water, sewer, and the like, so that each type of drawn locate mark is distinguishable from the other(s). The user input routine 560 may limit the display of such facilities by the type of work which is to be performed according to the instructions included within the user's assigned ticket. In addition to the marking of the underground facility locate marks on the aerial image, user input routine 560 may also include offsets from environmental landmarks that may be displayed on the image in, for example, English or metric units. Environmental landmarks may also be marked and/or highlighted on the aerial image. The user input routine 560 may also accept positioning information from external sources, such as a GPS-enabled device. The user input routine 560 may further include features to annotate the image with text and to revise user inputs by, for example deleting, dragging or pasting shapes. In one implementation, when the user zooms the image view in or out, user input (e.g., lines and/or shapes) that have been added to the original image may adhere to the changing image scale and remain in the original user-input locations.
The electronic manifest, which is a compilation of the aerial image and user inputs, may be saved as an image file. In another implementation, the user inputs may be saved in a mark-up format, including the geo-coordinates and underground facility type of each input.
In one implementation, the user device 210 may interface with a ticket management program for coordinating multiple tickets. The ticket manager routine 570 may facilitate such an interface. The ticket management program for coordinating multiple tickets may reside on the central server 220, for example, or on a separate server that is accessible to the user device 210. Generally, tickets may be stored on a central server and assigned to a user. When a user edits a ticket, the user may also have created an electronic manifest associated with the ticket. The ticket manager routine 570 may allow the user to synchronize the user's ticket cache with the company's central database and also synchronize the images and user input. The ticket manager routine 570 may copy images from the central server 220 to the user device 210 for new tickets, and will copy the user input from the user device 210 to the central server 220 for completed tickets. The ticket manager routine 570 may interface with the routines described above to correlate a user's assigned tickets with images for those tickets and download the images to the user device from the central server 220. The ticket manager routine 570 may retrieve the corresponding ticket number from the ticket management program when the user retrieves an image, or the ticket manager routine 570 may retrieve the image corresponding to an entered ticket number.
FIG. 6 provides a flowchart 600 of an exemplary process for creating an electronic manifest relating to underground facility locate marks. In one implementation, at least some of the blocks of FIG. 6 may be performed using user device 210 (FIG. 2). In another implementation, one or more of the blocks of FIG. 6 may be manually performed or performed by another device, such as central server 220.
The process 600 may begin with a user being dispatched to a dig area to be located. The user might be given a ticket that identifies what underground facilities the user needs to locate at the dig area. The ticket might specify, for example, the address or description of the dig area to be located, the day and/or time that the dig area is to be located, and/or whether the user is to locate the dig area for telecommunications, power, gas, water, sewer, or other underground facility. Based on information in the ticket, or other information about the dig area to be located, user device 210 in block 610 may associate the property address with a stored aerial image of the dig area. Such association may include associating the address with geographic location information, such as global positioning coordinates for the dig area extent (or boundary).
In block 620, the stored aerial image associated with the dig area to be located is retrieved from a cache of images and loaded into the user device 210. As previously described discussed herein with respect to FIG. 5, the cache of images may reside within the user device 210, the central server 220, a separate image server, or another storage device.
In block 630, the user may perform a locate operation to locate the underground facilities present within the dig area and mark the located underground facilities using a locating device and/or marking device, or a combined locating/marking device. For example, the user may use the locating device to identify an underground facility at the dig area, and may use the marking device to mark the underground facility with the appropriate marker (e.g., color paint, flag, or some other object). In certain implementations, information regarding the approximate geographic location of the applied underground facility locate marks may be gathered and stored electronically using a GPS-enabled device or other location identification device. The approximate geographic location of the underground facility locate marks may be determined, for example, by identifying the current geographic location of the OPS-enabled device as the user performs the locating or marking. In another implementation, a user may use a triangularization technique to determine the approximate geographic location of the underground facility locate marks. In yet another implementation, a user may determine latitude and longitude coordinates or some other measurement of a geographic location.
In block 640, information about the approximate geographic location of the underground facility locate marks may be added to the stored aerial image that was retrieved previously in block 620. The information about the approximate geographic location of the underground facility locate marks may be input by the user using an input device, such as input device 340 (FIG. 3) of user device 210. Additional aspects regarding information to be input by the user is discussed in more detail herein with respect to FIG. 8.
Still referring to block 640, information about the approximate geographic location of the underground facility locate marks may also be received directly from a GPS-enabled device, such as the GPS-enabled locating device or marking device used in block 630, and overlaid on the retrieved image. The user may use of a combination of received GPS information and manual entries to create an electronic manifest of the underground facility locate marks.
In block 645, if necessary, information about offsets of the underground facility locate marks from environmental landmarks may be added to the stored aerial image that was retrieved previously in block 620. As with the input of the facility locations in block 640, the location of the environmental landmarks may be input by the user using an input device, such as input device 340 (FIG. 3) of user device 210, or automatically input from a GPS-enabled device. The offset information may be automatically calculated or input by the user. Offset information may also be obtained by identifying selected environmental landmarks on the retrieved image and automatically calculating the distance from the selected environmental landmarks to the underground facility locate marks overlaid on the image.
In block 650, if necessary, information about the location of the underground facility locate marks may be converted to GPS coordinates. In block 660, the retrieved aerial image and information about the location of the underground facility locate marks may be stored in memory as a single combined image or electronic manifest. The electronic manifest may be stored as, or example, a digital image or an interactive electronic map. Additionally or alternatively, in block 670, the geographical coordinates of the underground facility locate marks may be stored in memory, such as memory 330 (FIG. 3), as a separate data set. The data set may be compiled as, for example, a database of GPS coordinates. In block 680, the combined image and/or separate data set may optionally be transmitted to a central location, such as central server 220 (FIG. 2).
FIG. 7 is a diagram of an exemplary data set that may be stored in memory 330 and/or transmitted to server 220. As shown in FIG. 7, a data set 700 may include a timestamp field 710, an underground facility identifier field 720, an underground facility location field 730, an environmental landmark identifier field 740, an environmental landmark location field 750, an other information field 760, a marking method field 770, a property address field 780, and a ticket number field 790. In another implementation, the data set 700 may include additional, fewer, or different fields.
Timestamp field 710 may include time data that identifies the day and/or time that the environmental landmark location was identified. The time data in timestamp field 710 is shown in FIG. 7 as 9:43 a.m. on Oct. 20, 2005—although any type of date and/or time code may be used. The information in timestamp field 710 may be useful in establishing when a locate operation occurred.
The underground facility identifier field 720 may include an identifier that uniquely identifies the type of underground facility that was marked. The identifier in underground facility identifier field 720 is shown in FIG. 7 as “power”—although any type of identifier may be used. Underground facility location field 730 may include geographic location information corresponding to an underground facility locate mark. In one implementation, the geographic location information may include a set of geographic points along the marking path of the located underground facility. The geographic location information in underground facility location field 730 is shown in FIG. 7 as N38°51.40748, W077°20.27798; . . . ; N38°51.40784, W077°20.27865—although any type of geographic location information may be used. The information in underground facility location field 730 may be useful in graphically presenting the underground facility locate marks on a map, and/or to verify that the locate operation was actually and accurately performed. Additionally, or alternatively, underground facility location field 730 may include geographic location information for multiple underground facility locate marks.
Environmental landmark identifier field 740 may include an identifier that uniquely identifies the type of environmental landmark being marked. The identifier in environmental landmark identifier field 740 is shown in FIG. 7 as “curb”—although any type of identifier may be used.
Environmental landmark location field 750 may include geographic location information corresponding to the environmental landmark identified in environmental landmark identifier field 740. The geographic location information in environmental landmark location field 750 is shown in FIG. 7 as N38°5140756, W077°20.27805; . . . ;N38°51.40773, W077°20.27858—although any type of geographic location information may be used.
Other information field 760 may store other data that may be useful, including user notes, such as distance information that identifies a distance between one or more environmental landmarks and one or more underground facility locate marks. Other information field 760 is shown in FIG. 7 as including “1.2 meters between curb and power line”—although any other data may be used. Additionally and/or alternatively, other information field 760 may include audio/voice data, transcribed voice-recognition data, or the like to incorporate user notes.
The underground facility owner field 765 may include the name of the owner/operator of the underground facility that has been marked during the locate operation. For example, in FIG. 7, the underground facility owner field 765 is shown as “ABC Corp.” Because multiple underground facilities may be marked during a single locate operation, it may be beneficial to associate each marked underground facility with a particular owner/operator
Marking method field 770 may indicate the type of marking used at the dig area to indicate the location of an underground facility. For example, in FIG. 7, marking method field 770 is shown indicating red paint. Property address field 780 may be the property address associated with the marking recorded in the data set 700. The property address field 780 may include, for example, the street address and zip code of the property. Other information in field 780 may include city, state, and/or county identifiers. The ticket number field 790 may include the ticket number associated with the locate operation, such as ticket “1234567” shown in FIG. 7.
In one implementation, the user device 210 may store multiple data sets corresponding to multiple underground facilities identified at a particular dig area. User device 210 may provide the data sets to server 220 in a batch—such as a batch corresponding to the group of underground facilities documented within the electronic manifest—or individually. The batch may be grouped together with other information generally relating to the locate operation, such as the name of the company responsible for performing the locate operation, the name of the locate technician, and the like. Additionally, or alternatively, the other information generally relating to the locate operation may be included in each data set.
FIG. 8 an exemplary diagram of a user interface 340 that may be presented via the user device 210. The user interface may be presented on a screen 800 that may be the screen of the user device 210, as described herein with respect to FIG. 2. The screen 800 may display a variety of graphical elements, including but not limited to: a map control 810, an address search panel 820, a locator palette 830, a navigation palette 840, a status bar 850, a menu bar 860, a service grid 870, and a scale bar 880.
Map control 810 generally may be the surface, or canvas, where images—such as an exemplary image 802—are displayed. The user may draw or input shapes “on top of” this surface using for example, the input device 340 of FIG. 3 to identify underground facility locate mark locations. FIG. 8 shows a stylus 804 as an exemplary form of input device 340.
The address search panel 820 may be used to identify images corresponding to a desired address. Panel 820 may, for example, accept a partial or complete address and allow the user to search for matches. If an excessive number of addresses match the search, then the size of the result set may be constrained. Address search results may be displayed which match the address search. The listed matches may serve as a springboard for displaying the image desired by the user. For example, when the user taps with a stylus 802 on an address match, the user device 210 may load the image corresponding to the selected address. As described above, this image may be stored locally on user device 210 or retrieved from central server 220.
Palettes may be generally defined as a toolbar or toolbars containing soft buttons or other controls that are grouped in some logical order. The buttons on a palette may duplicate the commands available on the menu bar 860. The locator palette 830 may allow the user to select the type of underground facility locate marks (e.g., electric, gas, water, sewer, telecommunications, etc.) the user will draw on the image 802. The locator palette 830 may also include a choice of various shapes or shades, such as freestyle, line, circle, rectangle, or other polygon that the user may select to draw on the image 802. In one implementation, the locator palette 830 may present a list of potential environmental landmark identifiers. In this case, the user may select an environmental landmark identifier from the list to overlay at the appropriate place on the aerial image 802.
The locator palette 830 may also include an offset tool that allows the user to mark the distance between, for example, an environmental landmark identifier and a drawn underground facility locate mark. Once the user has chosen the type of shape they wish to draw (freestyle, line, polygon, shading etc.) the application may track the user's movements to define the layout and location of the shape. The shape may be completed when the user terminates the drawing (for example, by lifting the stylus 804 or releasing the mouse button). A text label or other indicator may be added to the shape automatically based on the type of underground facility locate mark or environmental landmark selected (e.g., “electric” or “curb”) or may be manually added.
The navigation palette 840 may allow the user to zoom or pan the image 802. For example, the navigation palette 840 may include selections to zoom in, zoom out, or zoom to a selected section of the image. The navigation palette 840 may also include pan command buttons to pan left, pan right, pan up or pan down. Other selections that may be available on the navigation palette include buttons to alter the transparency of either the image 802 or the underground facility locate marks.
The status bar 850 may display information about the map control, such as the coordinates of the subject area, the coordinates of a cursor or stylus in relation to the image 802, and the image scale. The menu bar 860 may include an operating system element that allows a user to access commands, such as exiting the application, selecting what palettes or panels to display, or accessing online help.
The service grid 870 is shown as an exemplary “floating” window to show how the user interface for the screen 800 may operate in a typical operating system environment. The service grid 870 or any of the other graphical elements described in relation to screen 800 may be in a fixed or floating orientation. As underground facility locate marks are drawn on the map control 810, they may appear in a list in the service grid 870. Thus, the user may edit the properties of an underground facility shape using the service grid 870, as well as by selecting the shape in the map control 810. The service grid may include properties, such as the type, length, circumference, and material of the marked underground facility.
An electronic manifest of underground facility locate marks may serve several purposes. For example, the electronic manifest may provide significant improvements in accuracy and save time for the locate technician. Manual sketching is time consuming and imprecise. For example, with manual sketching, the general geographic features of the dig area location, i.e. roads, sidewalks, landscaping, buildings, and other landmarks, must be reproduced by the locate technician. Creation of an electronic manifest that includes drafting on retrieved aerial images may improve accuracy and eliminate drafting of these general geographic features.
Additionally, or alternatively, an electronic manifest of underground facility locate marks may provide a variety of data formats from a single user event. For example, electronic drafting creates data about the electronic manifest which can be reviewed without viewing the image. The type of marked underground facilities can be determined based upon the existence of different colors or other coding schema, length of marks for each underground facility can be approximated, and the existence and length of offsets detected. If available, the location of the marks can be cross-checked to the user's description or depiction of the area to be marked or excavated.
Additionally, or alternatively, an electronic manifest of underground facility locate marks may provide for easier dissemination and record-keeping. Electronic manifests can be associated with individual tickets and recalled electronically, avoiding the uncertainties and errors associated with manual filing systems. Furthermore, electronic manifests can be interrogated to ensure that the information recorded on the electronic manifest accurately comports with billing data or other information regarding the locate operation(s) performed.
Additionally, or alternatively, information from the electronic manifest regarding the distance between environmental landmarks and located underground facility locate marks may be used to verify subsequent locate operations or the accuracy of the electronic manifest. For example, if the information identifies an underground facility as running parallel to the curb at a distance of three meters, that information may be used to assess the accuracy or consistency of a subsequent locate operation at the same dig area or, upon inspection, the accuracy of the electronic manifest.
Additionally, or alternatively, information from the electronic manifest regarding the number and types of underground facilities may be used to estimate the scope of a subsequent locate operation to be performed at a dig area. For example, a large number of underground facilities may be indicative of an extensive (i.e., time-consuming) locate operation.
Additionally, or alternatively, information from the electronic manifest may be used by a quality control supervisor and/or damage inspector to verify the accuracy of the underground facility locate marks. For example, if the user who performed a locate operation indicated that an underground facility runs parallel to a driveway at a distance of two meters, then the quality control supervisor or damage inspector may use this information to verify whether the marks properly reflected the actual location of the underground facilities present within the dig area. Also information from the electronic manifest may be used to train a user and/or to perform quality control relating to a user's work.

CONCLUSION

Aspects of the invention as described herein enable retrieving from a database the appropriate aerial image of a specific geographic location, or dig area, where locate operations are to be conducted for underground facilities. The user may draft, on the retrieved image, a variety of features, including but not limited to (1) the type of underground facilities marked using an appropriate color or other coding schema, (2) the number of underground facilities marked within the dig area, (3) the approximate geographic location of each set of underground facility locate marks, and (4) the appropriate environmental landmark offsets for each set of underground facility locate marks. The combination of the retrieved image and additional information drafted by the user may be saved in a variety of formats as an electronic manifest. Other information regarding the specific geographic location of the locate marks and environmental landmarks may be incorporated into the electronic manifest using direct input from GPS-enabled positioning tools and the like.
The foregoing description is not intended to be exhaustive or to limit the description to the precise form disclosed. Modifications and variations are possible in light of the above disclosure or may be acquired from practice of the invention.
For example, certain information was described as being presented visually on a screen of user device 210. In other implementations, this information may be audibly provided to the user. Also, particular information was described as being input via an input device 340, such as a screen of user device 210. In other implementations, this information may be provided in other ways, such as by receiving inputs via input keys and/or buttons, by recognizing speech of the user, or by monitoring a condition of the user. More particularly, the input device 340 may be capable of capturing signals that reflect a user's intent. For example, the input device 340 may include a microphone that can capture a user's intent by capturing the user's audible commands. Alternatively, the input device 340 may interact with a device that monitors a condition of the user, such as eye movement, brain activity, or heart rate.
As another example, certain components, such as user device 210 and central server 220 were described as using an image cache. In other implementations, user device 210 and/or central server 220 may communicate with an image server (such as imager server 230) in real-time, so that no image cache may be required. In still other implementations, the user device 210 may, for example, communicate in real time with the central server 220.
As another example, it should be noted that reference to a GPS-enabled device is not limited to GPS systems only, and that any global navigation satellite system or other system that provides geo-spatial positioning may be used in implementations of the invention.
Also, while a series of blocks has been described with regard to FIG. 6, the order of the blocks may be modified in other implementations. Further, non-dependent blocks may be performed in parallel.
It will be apparent that aspects, as described above, may be implemented in many different forms of software, firmware, and hardware in the implementations illustrated in the figures. The actual software code or specialized control hardware used to implement these aspects is not limiting of the description provided herein. Thus, the operation and behavior of the aspects were described without reference to the specific software code—it being understood that software and control hardware can be designed to implement the aspects based on the description herein.
Even though particular combinations of features arc recited in the claims and/or disclosed in the specification, these combinations are not intended to limit the invention. In fact, many of these features may be combined in ways not specifically recited in the claims and/or disclosed in the specification.
No element, act, or instruction used in the present application should be construed as critical or essential to the invention unless explicitly described as such. Also, as used herein, the article “a” is intended to include one or more items. Where only one item is intended, the term “one” or similar language is used. Further, the phrase “based on” is intended to mean “based, at least in part, on” unless explicitly stated otherwise,

Claims

1. A method for generating a searchable electronic record of a locate operation performed by a locate technician, the locate operation comprising identifying, using at least one physical locate mark, a presence or an absence of at least one underground facility within a dig area, wherein at least a portion of the dig area may be excavated or disturbed during excavation activities, the method comprising:

A) electronically receiving a digital image of a geographic area comprising the dig area, at least a portion of the received digital image being displayed on a display device;

B) adding to the displayed digital image at least one digital representation of the at least one physical locate mark applied by the locate technician during the locate operation so as to generate a marked-up digital image including the at least one digital representation of the at least one physical locate mark; and

C) electronically transmitting and/or electronically storing information relating to the marked-up digital image so as to generate the searchable electronic record of the locate operation.

2. The method of claim 1, wherein B) comprises:

adding, via a user input device associated with the display device, the at least one digital representation of the at least one physical locate mark to the displayed digital image, so as to generate the marked-up image.

3. The method of claim 2, wherein the user input device comprises a stylus.

4. The method of claim 2, wherein the user input device comprises a touch screen.

5. The method of claim 2, wherein prior to B), the method comprises:

displaying a palette on the display device; and

selecting a type of underground facility locate mark from the palette;

and wherein B) further comprises:

adding the at least one digital representation based on the selected type, such that the at least one digital representation includes at least one attribute representing the selected type of underground facility locate mark.

6. The method of claim 5, wherein the at least one attribute comprises a color of the at least one digital representation.

7. The method of claim 2, wherein A) comprises:

electronically receiving the digital image and at least one electronic marking tool application to facilitate B).

8. The method of claim 1, further comprising:

D) electronically receiving geographic location information; and

E) associating address information with the geographic location information,

wherein the information electronically transmitted and/or electronically stored in C) comprises the address information.

9. The method of claim 8, wherein the address information includes at least one of a street address, a zip code, a street name, and a city.

10. The method of claim 8, wherein the geographic location information corresponds to the dig area.

11. The method of claim 8, wherein the geographic location information corresponds to a location of a user of the display device.

12. The method of claim 8, wherein D) comprises:

electronically receiving the geographic location information as geographic coordinates from a GPS-enabled device.

13. The method of claim 1, wherein the information electronically transmitted and/or electronically stored in C) comprises time data to establish when the at least one physical locate mark was applied by the locate technician.

14. The method of claim 1, wherein the information electronically transmitted and/or electronically stored in C) comprises a name of the locate technician.

15. The method of claim 1, wherein the information electronically transmitted and/or electronically stored in C) comprises a name of a company responsible for performing the locate operation.

16. The method of claim 1, wherein the information electronically transmitted and/or electronically stored in C) comprises a set of geographic points corresponding to the at least one physical locate mark.

17. The method of claim 1, wherein the displayed digital image comprises an identification of an environmental landmark location, and wherein the information electronically transmitted and/or electronically stored in C) comprises time data that indicates the day and/or time that the environmental landmark location was identified by the locate technician.

18. The method of claim 1 wherein the displayed digital image comprises an identification of an environmental landmark, the method further comprising:

D) automatically calculating a physical distance between the environmental landmark and the at least one physical locate mark based on a representative distance between the identification of an environmental landmark and the at least one digital representation of the at least one physical locate mark in the displayed digital image.

19. The method of claim 18, wherein prior to D), the method comprises:

adding to the displayed digital image the identification of the environmental landmark.

20. The method of claim 1, wherein the digital image received in A) comprises an identification of an environmental landmark.

21. The method of claim 1, further comprising:

receiving an input indicative of a condition of a user of the display device.

22. The method of claim 21, wherein the input indicative of a condition of a user includes information concerning one or more of: brain activity of the user, a heart rate of the user, and eye movement of the user.

23. The method of claim 1, wherein C) comprises:

electronically transmitting the information relating to the marked-up digital image to a central server.

24. A computer-readable medium encoded with instructions that, when executed on at least one processing unit, perform a method of generating a searchable electronic record of a locate operation performed by a locate technician, the locate operation comprising identifying, using at least one physical locate mark, a presence or an absence of at least one underground facility within a dig area, wherein at least a portion of the dig area may be excavated or disturbed during excavation activities, the method comprising:

A) electronically receiving a digital image of a geographic area comprising the dig area;

B) receiving user input regarding a geographic location of the at least one physical locate mark applied by the locate technician during the locate operation;

C) rendering a screen display comprising at least a portion of the received digital image and at least one digital representation of the at least one physical locate mark, wherein the at least one digital representation is positioned with respect to the at least a portion of the received digital image based at least in part on the user input received in B).

25. An apparatus for facilitating generation of a searchable electronic record of a locate operation performed by a locate technician, the locate operation comprising identifying, using at least one physical locate mark, a presence or an absence of at least one underground facility within a dig area, wherein at least a portion of the dig area may be excavated or disturbed during excavation activities, the apparatus comprising:

a communication interface;

a display device;

a user input device;

a memory to store processor-executable instructions; and

a processing unit coupled to the communication interface, the display device, the user input device, and the memory, wherein upon execution of the processor-executable instructions by the processing unit, the processing unit:

controls the communication interface to electronically receive a digital image of a geographic area including the dig area;

controls the display device to display at least a portion of the received digital image;

acquires user input from the user input device, the user input relating to a geographic location of the at least one physical locate mark applied by the locate technician during the locate operation;

generates a marked-up digital image including at least one digital representation of the at least one physical locate mark based at least in part on the user input; and

further controls the communication interface and/or the memory to electronically transmit and/or electronically store information relating to the marked-up digital image so as to generate the searchable electronic record of the locate operation.

26. The apparatus of claim 25, further comprising a location identification unit to determine a geographic location of the apparatus.

27. The apparatus of claim 25, wherein the user input device includes one of a stylus and a mouse.

28. The apparatus of claim 25, wherein the user input device includes one of a keyboard, a keypad, a touchpad, and a touch screen.

29. The apparatus of claim 25, wherein the user input device includes one of a microphone to capture audible commands, and a camera.

30. The apparatus of claim 25, further comprising a sensor to monitor a condition of a user using the apparatus.

31. The apparatus of claim 25, wherein the processing unit:

controls the communication interface to electronically receive a digital image comprising an identification of an environmental landmark.

32. The apparatus of claim 25, wherein the processing unit controls the communication interface to electronically receive the digital image and at least one electronic marking tool application to facilitate acquisition of the user input from the user input device.

33. The apparatus of claim 25, wherein the information electronically transmitted by the communication interface and/or electronically stored in the memory comprises at least one of:

the marked-up digital image; and

geographic location information corresponding to the at least one physical locate mark digitally represented in the marked-up digital image.