Webinar: Neustar's Network Geography Analysts — The Human Element of IP Geolocation

Geolocation and Geology

Thanks Susan. So we'll start off with a little historical context. Maps have been around forever, and they've always been created by humans really up until the last 10 years or so. We've gone all the way from tablets to hand drawn maps.

And at every stage, whether it was creation of the map itself, or for example, the map that we're looking at now probably said here there be dragons off of the edge of the world. But then at some point, somebody crossed the edge of the world and discovered there were no dragons, and so they updated it. And that's kind of what we're going to talk about today.

Because today, when people do digital map updates, such as Google's, it's really important that you have humans involved in the creation of the maps and also in gathering feedback. Because although machines can automate certain parts of it and make things go very fast, machines do not deal well with ambiguous situations. And that's where humans come in, and that we do quite well with that.

Customer Data Feedback Loop

One of the things that accurate geolocation requires is an understanding of what your services are, who you're going to be providing to. So we're at Neustar, one of the things that we try to do is have an advanced architecture foundation and collection methodology. That's the foundation for everything.

We know who our customers are, what use cases they're going to use it for, how the data will be used. That's always important. For example, you don't want to-- if I go back to our map, nautical map, you don't want to give a nautical map to a mountain climber. That's an inappropriate map because it doesn't tell him anything about elevation changes, et cetera.

We need tools that are designed to allow both the map makers themselves and the consumers to get insight about what it is that they're being presented with. And what we're going to talk about today is really about how feedback and collaboration with customers and users of these maps can be very important in making the maps more accurate.

Importance of Fresh Data

And lastly, in order to incorporate that feedback in, you want to make sure that it goes in a timely fashion because it doesn't really do your end users a lot of good if you, for example, Google Maps, if a bridge goes out and Google Maps is still routing people across that bridge 24, 48 hours later, that's potential for a lot of injury and death. So you want make sure that your maps are fresh and that carries over as well into the IP address space.

Geolocation Agenda

So, what we're going to talk about, our agenda is basically-- I'll tell you a little bit about IP geolocation, about geo-feedback. We'll tell you about why geo-feedback's important. Paige will talk about how her network geography analysts improve our data both on the reference side and the data that we get back from customers. She'll talk about the process, how that works. And I'll go through a few case studies where we work with customers to-- they helped us improve the data, and then in turn help them provide better support and service for their customers.

So let's go on to the next slide. So is IP Geolocation? So what's an IP address? An IP address is what's assigned to a device that's on a network. And in internet terms, we have IPv4 and v6, which accomplish the same things. IPv6 is a much larger address space that theoretically we will never run out of.

But the important thing that many customers want to know or many of our customers users is, OK, I have an IP address, it's mapped somewhere in the network world, how does that relate to the physical world? And so IP Geolocation attempts to answer that question. It's a process of estimating the placement of an IP address on a physical map by country, state, city, et cetera.

It's probabilistic. There's no real truth set that guarantees that any IP address is any location.

Sometimes we get information from internet service providers or from the registry services that helps us make broad associations of where IP addresses are. But in the end, we are dependent on a lot of human interaction with our algorithms and with our data to help define placed data such as at the city level.

More About Geo-feedback

Geo-feedback is the term that we use here at Neustar to refer to when a customer or a partner gives us information about when we got it wrong or a piece of our data-- well, not everything was wrong, but maybe a piece was wrong, maybe we've gotten zip code incorrect, but the city was right, et cetera, and that helps us in turn make our data better. And we try and take that data and feed it back into our process and then turn that around and publish it within a week's time. So that's very fresh and it allows not only that customer but other customers to take advantage of that more accurate data.

So why is geo-feedback useful? Well, as I mentioned, it lets customers get more accurate data. It takes something that is approximate, which is what our algorithms and what our systems do, even though we have humans overlooking it.

And when we get exact feedback from an end user, assuming that we trust that feedback-- and then we'll talk about that in our process-- but then we can-- it helps actually not only improve that particular IP address location, but also IP addresses that are related nearby, so a block of IP addresses can now-- now we have an additional data point to help us place those IP addresses. So now what we're going to do, I'm going to hand it over to Paige. And she's going to talk a little bit about the geo-feedback process.

Thanks Steve. Hi, this is Paige Enoch, and I'll be going over the geo-feedback process. So at the core of this process is our team of network geography analysts, or NGAs. They really are the backbone of our data analysis system. They have combined decades of experience looking at IP Geolocation data and really understanding how networks allocate network-- allocate their ranges and where those ranges might be.

So a few of the things that they focus on are they review reference data before it's incorporated into the system. They will perform analysis before data is released to make sure that everything is looking good and looking accurate. And finally, they'll also validate the geo-feedback requests for inclusion in the data release.

And these geo-feedbacks are critical for customers and for Neustar. They can resolve user issues faster, they can improve call center efficiency, and they can increase user retention. If there's a user that's been planning to binge watch a TV show with friends for several weeks, and suddenly they can't access that content, they might be getting a message that they're in the wrong location, they'll likely call into a call center and say, I'm actually in this place, can you look into this?

Another user might be trying to play the lottery or access a gambling site. They might also be in an approved location. And they might email a support representative and say, I can't access this.

What's going on?

And finally, a really die-hard fan might have been looking forward to a live event. And when they go to access the event, they can't access it due to a geolocation restriction. So they might file an online support case as well. So each of these customers might also post their dissatisfaction on social channels. And if the issue reoccurs, they might not renew their subscription.

NGA Review Process

So how are these issues escalated to the NGA team for review? Each of the issues we just discussed might be escalated to a customer support representative. And from there, that representative can go to our Geo-feedback Portal where they can enter information about what the customer is seeing, where the customer is saying that they are, and really just other supporting details about any discrepancies that they might be seeing.

From there, those issues are assigned to an NGA for review. The NGA will then perform a detailed analysis of that IP. So they might consider the location, of course, but they're also looking at things like connection type, routing type. They might also be considering host names, or net name, or organization locations for that range.

They're also applying their really extensive knowledge about networks and about ISPs during this process. For example, there's a large cable provider that generally tends to name their host names after the nearest large metro area. And in general, that's a really great approach and will help us get those networks correct.

But in some areas, that can be problematic. For example in the Missouri-Illinois region, that is a border region around St. Louis. So if the host names are indicating St. Louis, then customers just across the border in Illinois might be getting blocked because we might be geolocating them to Missouri.

So the NGA can look at these networks and really understand exactly where they think these IPs should be geolocated. So they will make a determination and a decision for where the IP should be geolocated. They will then add that update to our IP GeoPoint database. They will notify the customer of the decision that they made, so whether they changed it to be closer to where the customer is saying they are, or if they left it unchanged.

And finally, all customer portals will be updated with that new information. And they'll be able to see those updates. So now, I will hand it back over to Steve to discuss some case studies.

OK, so now we're going to go through three case studies where we talk about how our network geography analysts worked with customers and provided them with more accurate IP Geolocation data. Let's go to the first slide.

Case Study: Streaming Customers and Geolocation

This case study has to do with a streaming video movie/TV content provider who has restrictions from the content owners themselves about where they can stream that content to. So for example, certain countries are allowed, certain countries are not allowed, and one of the things that this customer does is they look at our geolocation data to see the country-level information. They also look at our metadata that we have about connections that, for example, is it coming over VPN or anonymizer, which are types of connections that are normally used to hide where the end user's really coming from.

So in those cases, you might have an end user who's in a blocked location trying to view something. But in some cases, you might have a user who's in an allowed location for whatever reason the system is not allowing him to see it. If it turns out that the root cause is because the geolocation data has that user in an incorrect location, then we can make the change to that.

And so the way that that change is implemented is that this customer sends us what we call bulk geo-feedback in the form of a log file that has multiple requests for changes. It says, here's the IP address. Here is the location that we believe the end user is actually at. Please review this and determine if it's accurate or not accurate.

And that's what our NGAs do. And then that information is put back into the system and published approximately four days later. And meanwhile, this customer gives their end users a token that allows them to view the content temporarily until the next published thing comes out.

And then if that customer was to call in again, that end user, they would know whether or not the data had been accepted or rejected, and they could then say, well yes, it looks like you're actually not allowed to view this data. So this has been a very useful tool in their hands and to reduce the cost that they have in their customer support center because now they're able to automate a lot of handling of the calls and have a higher customer support rating.

Case Study: MLB & Local Blackout Zone

So now going to go on to the second case study. And this case study is MLB.com, Major League Baseball. And they not only stream baseball, they stream other sporting events as well, for example, for hockey.

So they have a more specific granular geo-fencing requirement, not only at the country level, but actually down at the city level. And it's expressed in the form of a Nielsen DMA or postal code. But essentially, it has to do with when a sporting event is in a certain location, and it's not sold out, they are not allowed to stream that content to that location because they want to encourage people to go to the stadium. And so they use our geolocation data to determine whether it's a blackout zone or not.

And the interesting thing about this use case is that our NGAs worked with MLB beforehand to help MLB understand what data we could provide to them and how they might best use it to meet their requirements for doing the blackouts and getting as granular as possible. And so in this case, the NGAs were very useful in getting things set up. And so now we have a very smooth system with them. MLB submits both what we call on-demand geo-feedbacks and bulk feedbacks. And it's been very useful for them and for similar customers.

Case Study: A Risky Lottery Ticket

So now let's go into our last case study. This is an online gaming/gambling scenario. It's actually a lottery provider for a certain region of Canada. And they have restrictions that prevent citizens who are not region-- members of that region from actually participating in the lottery. So if they're outside of Canada, if they're outside of that region of Canada, they're not allowed to play.

And they use our geolocation information to make sure that they do not get hit with fines. Because if they allow someone to play and win who's not from their region, they will get hit with fines. So this helps reduce risk on their end in addition to making the lottery fair for all the citizens of that particular region. This lottery, they also submit bulk weekly geo-feedback files. And we turn that around within four business days so that they are ready to go before their next lottery happens, which is usually on a weekly basis.

So those are the three case studies. We have others that we can talk about if you're interested. Contact Neustar for that. But now let's go on and see if we've gotten any Q&A from our viewers.

IP Geolocation Q&A

Thank you, Steve. Now we'll take questions from our audience. If you have a question, please type it into the Q&A Pod and click Submit, and we'll get to it as soon as we can.

We have been seeing your questions as they come in. If you haven't already submitted your question, please do so now by typing it into the Q&A Pod and clicking Submit. So let's take a look.

Q: Why would there be errors in determining the accurate location of an IP address?

We have one here. Why would there be errors in determining the accurate location of an IP address? OK, so the issue is here that IP addresses are really not allocated based on geography to begin with. They're allocated based on network parameters that are required.

So what we're doing with IP Geolocation is really something that's very probabilistic. We're doing a best guess based on the information that we collect. So in most cases, were accurate. But in some cases because of the way networks are allocated, we're 100% accurate I don't know if Paige if you want to add anything to that.

Yeah, it's a really interesting problem. There are about 2.7 billion routable IP addresses. And those locations change pretty regularly and pretty frequently. So we certainly do our best to stay up to date and get things in the right place, but ultimately for some IPs, we will get that location wrong.

And we're definitely not alone there. Any provider will get some percentage of the IPs incorrect. But where we really excel is with our team of NGAs who can review those discrepancies, make an informed decision, and make those changes quickly.

Q: What data sources do you use?

OK, great. We have another one here. What data sources do you use?

So Neustar actually has a number of internal data sources. We own and operate domain name service, DNS service, et cetera that provide us with some information. We have external sources that we can collect publicly available. There's some sources that we pay for.

And all of these are mixed together what we call our data synthesis process where we do some weighting, comparing one source versus another. Some sources are typically known historically to be more accurate than others. But we use everything to try and get a-- to come up with a single answer at a given point in time for where that IP address is located. Paige, anything on that one?

Yeah. So we have a really robust data collection network, so we're constantly recollecting data and making sure that everything is up to date. And our other third-party sources in there also refreshed constantly. So we are really consistently staying on top of changes.

Q: How do you handle issues related to personal information?

Great. We have another one. How do you handle issues related to personal information?

OK, so Neustar is very concerned about maintaining the privacy of individuals, especially there's the new GDPR regulations in Europe. So we collect information, but here on the IP Geolocation data, we do not get that information down to an address level. What we go to is the city level and the zip code level. So we can't actually associate an IP address with a person, and that puts us in full compliance with protection of individual information.

Q: Do you provide global geolocation data? And are there any countries that are not included in your data set?

Great. And we have another one. Do you provide global geolocation data? And are there any countries that are not included in your data set?

Absolutely. Our data set is global in nature. There are some countries where we have more data and some countries where we have less data, but we do have coverage for all of the countries in the globe. And that goes for both IPv4 and IPv6.

As far as I know, we're not missing in countries. Is that right Paige? Paige is shaking your head and says, no, we are not missing any countries.

Q: Is IPv6 different from IPv4?

OK, that brings up another one that I see here. Is IPv6 different from IPv4?

Yes. IPv6 is very challenging. There are like a quadrillion or 18 quadrillion IPv6 addresses. There's only somewhere in the neighborhood of four billion-- only four billion IPv4 addresses. And IPv6 is designed to replace v4, like we're never supposed to run out.

And the challenge here is that there are so many addresses that at this stage in the rollout of IPv6, not all of them, are in use. So it takes a lot of information and smart algorithms to be able to identify which IP addresses are in use, scan them, get them into the database, et cetera. So in that way, IPv6 is more challenging.

But we're actually doing a pretty good job. We've more than doubled our coverage. We now have 99.9% coverage of all allocated IP addresses. And in terms of assignments, it's increasing every day. So we're very happy with the way our IPv6 rollout's going.

Q: How do you discover anonymizer VPN IP addresses?

Great. And we have one more here. How do you discover anonymizer VPN IP addresses?

Yeah, so I can take this one. There are two types of anonymizer IP addresses. There are public addresses and private addresses.

So public proxies are ones that are freely available, anyone can use them. And we collect and test those in are very automated way. For the private IP addresses, those are primarily used by VPN services or TOR. So we research and collect those VPN addresses in a more manual and specialized way.

Great. I think that's the end of our questions right now. So, I would like to thank both of our presenters as well as our audience for your participation. And we look forward to meeting you at our next Neustar webinar. Have a fantastic day.