It is not a stretch to say that if you don’t understand the Internet, then you can’t really understand digital experience delivery today. In other words, depth of understanding of the Internet is essential and critical to sound Digital Experience Monitoring. As the routing protocol that runs the Internet, BGP is a key piece of this puzzle that helps you understand how your customers get to you, which means that BGP visibility is very important if you intend to have operational insights for any business-critical app or service that you are either offering or consuming over the Internet.
But visibility and monitoring as terms can be a bit vague. And there are varying claims to BGP visibility or monitoring out there. So in this blog post, we’ll unpack briefly how BGP works and the implications for monitoring, what types of ‘BGP monitoring’ exist out in the wild and how to distinguish them, key capabilities you should look for in BGP monitoring, and a little hack on telling the difference between serious BGP visibility and the poseurs.
The Shortest BGP Overview Ever
For those network operators who are deeply knowledgeable about routing, forgive the drastic simplification of a complicated process for the Internet’s control plane. The BGP (Border Gateway Protocol) is a path vector routing protocol that essentially concerns itself with two major functions:
- Establishing routed peerings (communication sessions) between Autonomous Systems or ASes (networks that have registered to participate in the BGP fabric of the Internet) so they can exchange routing information (how to get across the Internet to various prefixes (network addresses). There are over 63,000 ASNs as of the drafting of this post.
- Propagating routes to IP prefixes (network addresses) across all those AS. Routes are defined not as paths through individual routers, but as paths through Autonomous Systems. So, when you look at a BGP routing update message, you’ll see a sequence of AS Numbers (ASNs) which forms an AS-PATH, corresponding to a specific prefix.
A BGP routing update can contain multiple AS-PATHS for a prefix, along with multiple AS-Path attributes. As of the drafting of this blog post, the IPv4 BGP routing table for the Internet currently contains 768,385 prefixes.
Routing Data Hinges on Perspective
In the Internet, unless there is some type of filtering happening (see great firewall), the assumption is that you can reach anywhere in the Internet from anywhere. However, the path your traffic takes to a given location will differ based on where you’re coming from. Furthermore, with the vastness of the Internet, it’s possible for a single routing vantage point to introduce spurious routes. That’s why if you want to understand Internet routing you need to get and intelligently process a lot of different perspectives from different ISPs for global visibility to get close to ground truth on Internet routing behavior. For example, ThousandEyes collects active routes and updates from many dozens of BGP feeds around the world for more complete global reachability and prefix visibility. Breadth of data collection provides perspective.
BGP Monitoring is Getting More Popular—Thank Goodness
One of the things that’s been interesting but ultimately great to see is the rising awareness that understanding Internet performance is critical for sound network performance monitoring (NPM) and digital experience monitoring (DEM). I say rising because if you go back a couple of years you’d see some NPM vendors positively bashing BGP monitoring, and most other studiously ignoring it. But with so many organizations using the cloud to build apps and services, offering customer digital experiences, consuming SaaS, and modernizing your WAN, the influence of the Internet has become too big to ignore. The most competitive digital organizations have built or are building expertise in global connectivity, interdomain routing and its complex interactions with their internal routing policies, BGP policies, managing ISPs. All this expertise goes towards tuning operations to get the best performance. As a result, even skeptical vendors are becoming BGP visibility converts.
Five Implementation Types for BGP Monitoring
Sadly, while BGP has a clear definition as a protocol, the meaning of the term “BGP monitoring” can be highly variable depending on who’s claiming it. So it might be helpful to outline five different implementations by which BGP routing data is offered as “visibility”.
1. Light integration of BGP routing attribute data into network-layer paths. Some monitoring products essentially take a feed of BGP attribute data, then enhance network-layer path information. Now, this is where we get into semantics. It’s possible to simply label various nodes in a Layer 3 path with the names of the ASN they’re in by doing prefix lookups against a single BGP routing feed. But this is a real stretch to call “BGP monitoring,” or “BGP visualization.” After all, you can’t say that you’re monitoring BGP routing or visualizing it when you can’t look at prefixes or AS-PATHS.
2. Third-party open source tool linking. Some monitoring products provide a link to external, open source tools such as RIPEStat BGPlay, as seen in figure 1. The upside to this approach is that you get to use a cool tool where you can do some BGP prefix analysis. The downside is that it’s not meant for ongoing monitoring so much as snapshot views, isn’t integrated with the rest of the product workflow, and isn’t really meant for business use.
Figure 2: RIPE NCC BGPlay output for the prefix containing the quad 8 DNS service address.
3. BGP traffic analysis. This is an interesting concept, whereby traffic flow data is enhanced by prefix matching the source and destination IPs then mapping to BGP attributes for those prefixes, such that you can see traffic volumetrics from a source AS to destination AS, and even via transit AS. That’s definitely interesting and useful if you’re moving large volumes of service traffic to the Internet. However, the focus of this capability is traffic analytics rather than monitoring or visualizing how BGP routing itself is working.
4. Standalone BGP visibility toolkits. There are some open source and commercial tools that perform BGP prefix monitoring on a standalone basis. These tools are used by some large organizations, but generally, they are difficult for IT teams to use for troubleshooting application and service issues because they typically are offered as data feeds. That means the IT organization needs to integrate that data and perform its own correlation against other tools in the stack. Furthermore, these data feeds can be filled with routing issues from the most unstable fringes of the Internet, creating a ton of useless noise. This is legit BGP monitoring, but it’s just not very useful for business purposes by the average IT team.
5. Integrated BGP route monitoring. Integrated BGP route monitoring means directly pulling collected global routing tables and updates on a frequent basis and integrating BGP prefix monitoring, reachability information and visualization of ASes, AS paths, path lengths, etc. with other aspects of digital experience monitoring (DEM) so that it delivers contextually useful insights in real-time for app and service operations visibility. As mentioned above, you need that BGP routing data from many points on the Internet and to use intelligent algorithms to create a sufficiently accurate perspective.
Key Capabilities to Look for in BGP Monitoring
Given that there are so many products that claim to offer insight into Internet routing behavior but in reality provide something that is quite a bit less, it might be helpful to know the key capabilities you should be looking for to support digital experience monitoring use cases. The following metrics and visualizations should be available on a time-series, historical basis:
- Metrics:
- Prefix reachability
- Prefix path changes
- Prefix updates
- Independent AS-PATH visualizations, ideally linked to higher level monitoring against an app or service URL for example.
- Ideally all prefixes related to monitoring test connectivity to that URL are automatically detected
- Cross-layer correlation
- To be truly useful, BGP routing data should be time-series correlated to other layers of data including network layer paths, end-to-end network performance metrics (packet loss, latency, jitter), and app-layer metrics (response time, page load, etc.)
Like with most monitoring, visualization is critical for ease of use. At ThousandEyes, we’ve made significant investments, including patents, into our visualizations of network paths and BGP routing. Read more on our latest BGP visualizations here.
A Quick Keyword Hack to See if BGP Monitoring is for Real
Here’s an easy way to see if you’re looking at real BGP monitoring or a fluffy marketing copy version. Search the product or vendor name, plus “BGP” and “prefix.” Then compare the results. You’ll quickly see who has the goods, who is regularly writing about real-world BGP issues relating to digital experience delivery; and who is providing insights into BGP routing and the Internet to the industry as an expert.
ThousandEyes Leads BGP and Internet Visibility
ThousandEyes has offered BGP monitoring and visualization for years, and has even pushed beyond routing to deliver collective intelligence-based Internet traffic outage insights and alerts. You can read some of our blog posts on BGP routing issues such as the route leak that hit users of Google Cloud services, and the BGP hijack that took down a portion of the AWS Route 53 DNS service. Or check out some coverage on our commentary around the recent Facebook outage, where we identified the issue early on as internal to Facebook, and corrected mistaken notions of BGP routing as a root cause for a major Facebook outage.
If you know your team needs BGP visibility to effectively deliver digital experiences to customers or employees, contact us and we’ll set up a deep dive, answer your questions on BGP routing and security issues and how to detect them.