If you'd like to experiment with running your own AS in private address space, connecting to a friendly network of geeks over wireguard tunnels, check out DN42 https://dn42.dev/Home.
It's a great way to explore routing technologies and safely experiment with your own AS, running the same protocols as the "real" Internet, just in private space.
If you do get set up, give me a shout (https://markround.com/dn42), I'd be happy to peer with you if you want to expand beyond the big "autopeer" networks :)
I think HN tends to undo all caps words unless it's an acronym HN specifically recognizes. Guessing BGP, GRE, and FreeBSD are understood but AS is not.
It’s too late now, but when submitting a post the poster has a window of time to edit the title. Useful for example when HN auto-edits to capitalisation get some words wrong. When you edit the title, those auto-edits are not applied to your edited title.
I was hoping with IPv6, getting an address space as an individual would go back to how it was in the early IPv4 days, but alas you need to be a multihomed individual with tons of usage instead of just a sophisticated netzien that wants to own their block.
One of my customers was handing out /64s for a while but it was more hassle than it was worth. I only ever saw one residential customer use it, and he was just smart enough to cause problems.
Its one of those things that there needs to be strong consumer demand for, or it will just never happen tbh.
From our perspective, what we want more than anything in the universe is to never do NAT or DNS ever again. I would much rather maintain a billing system indicating you rent a small block of IPV6 space, with a nice little static route, over maintaining never ending NAT and DNS logs for the benefit of police forces who cant shit without collecting every micron of data. But NAT is basically security these days, and theres a negative driver in exposing customer routers directly to the internet (in that, if it even supports v6 its likely to be rooted) Customers will leave if telcos do things properly, and theres literally zero reward for being nice about it.
Interesting, my two ISPs (one in Belgium, one in France, not business ISPs) hand out fixed /48 blocks to every customer. As far as I know, that's what RIPE recommends, they actively discourage from assigning longer prefixes than /56.
The modems they provide handle it without needing anything special from the customers. The devices get IPv6 addresses from this prefix, and are firewalled by default. It's pretty simple so I'm not sure what could go wrong there.
Actually, they don't want to pollute the internet routing table with routes that are fully subsumed into other routes. The effect on address ownership is a side effect.
RIRs, not ISPs, allocate addresses at the top level, they make money on each address allocation, and they still won't allocate addresses to you if you don't multihome because they have a duty to conserve resources.
When you get PI addresses your LIR/ISP just passes your data on to the RIR.
Just like many industries there's a retail side and a wholesale side. You're asking to get a wholesale product from a retail channel. If you become a wholesale customer you can get what you want, for a price.
I don't want an address, they should be cheap, meaningless (sans routing, the longer the common prefix, the closer geographically you should be) and not conflated with identifiers.
I just want a way to do public-key based discovery. I'm not sure if wireguard + DHT would do though as it'd also mean that it's easy to track your PK (and maybe you through your devices/services announced with PKs).
Maybe you can announce your IP in a neat encryption scheme that adds some privacy without increasing costs too much?
Anything in your private network (even if it goes over public internet) should be encrypted and locked up anyway. Something like Wireguard or Nebula only needs a few (maybe just one) publicly accessible address. Inside the overlay network, it's easy to keep IP addresses stable.
Anything public-facing likely needs a DNS record, updatable quickly when the IP of a publicly accessible interface changes (infrequently).
The realistic point is to have your own abuse email contact, to evade the banhappy policies that most server hosts have even when you did nothing wrong. Usually they suspend your account if you don't reply within 24 hours, even if the complaint is obvious nonsense.
Even fast LACP needs three seconds and that's on the same collision domain.
How does BGP actually detect a link is down? Keep alive default is 30s but that can be changed. If you set it to say one second, is that wise? Once a link is down, that fact will propagate at the speed of BGP and other routing protocols. Recovery will need a similar propagation.
Depending on where the link is, a second can be a "life time" these days or not. It really depends on the environment what an appropriate heart beat interval might be.
Also, given that BGP is TCP based, it might have to interact with other lower level link detection protocols.
It can get a bit hardware dependant but getting <50ms failovers from software based BFD in BIRD or FRR is fairly easy, and I've tested down to < 1ms before with hardware based BFD echo. ~50ms is the point at which a user making a traditional VOIP call won't notice the path switch.
You can get NIC's for computers (like most Nvidia/Meallanox or higher end Broadcom/Intel NIC's that do hardware BFD, and its obviously included in higher end networking kit.
You then link the BGP routes to the health of the BFD session for which that path is the next hop, and you get super quick withdrawls.
I.e. bird detects interface failure but this affects only your side of decision making.
For bidirectional failure detection you do BFD with BGB. BFD default timers are 3 times 30 ms, iirc.
I have both my own multihomed ASN and operate my own nameservers. The latter has usually been about as fast for failover overall in practice. BGP may look to converge near instantly from your 2-3 peer outbound perspective but the inbound convergence from the 100k networks on the rest of the internet is much slower and has a long tail very akin to trying to set your DNS TTL to 0 and having the rest of the internet decide to do it slower for cache/churn reasons anyways.
The bigger problem, and where BGP multihoming is most handy, is it's just so much easier to get a holistic in+out failover where nothing really changes vs in DNS where it's more about getting the future inbound stuff to change where it goes. E.g. it's a pain to break an active session because the address had to change, even if DNS can update where the new service is quickly.
The long tail of routers receiving your update doesn’t matter. Once the common transit networks get it, that’s where the rest would dump the traffic to reach you anyway. The only time slow propagation to the edges matters is the first time announcing a prefix after it has been fully withdrawn.
Using the wrong route to get the packet in your general direction still gets you the packet as long as it hits an ISP along the way that got the update.
We could fully drain traffic from a transit provider in <60s with a withdrawal with all of the major providers you get at the internet exchanges. If you weren’t seeing that your upstream ISPs may have penalized you for flapping too much and put in explicit delays.
<60s sounds about right as a general safe estimate. I just mean people should expect 1-2ish orders of magnitude more than <1s from a downed link with internet BGP upstreams in a multihomed situation.
If it were flap suppression/slow peer detection/"the dunce bucket" there wouldn't be a long tail of convergence - it'd just be nothing until all at once. This also isn't something I've only seen on my personal AS alone, it's what I've come to expect in many enterprise cutovers while previously working at a network VAR. The personal AS is however much more carefree to move around to different random providers on a whimthough of course :).
I found some data from an oldish post by benjojo https://blog.benjojo.co.uk/post/speed-of-bgp-network-propaga... which confirm various tirr 1s do propagate updates across their networks very fast (<2ish seconds) while others certainly do not. Notably, Level 3 (now Lumen) is the largest BGP presence by prefix count and was the worst tested in the list - starting to apply at ~20s after to finishing at ~50s after. This was for announce specifically, which should be the clearer case.
Honestly it's not free but it's really not that expensive. With RIPE it's about 75€ per year for the ASN and being multihomed is not really a problem, there are multiple services that will let you announce through them for free or very cheap. You don't have volume minimums.
I do agree it should be simpler, but it is accessible to individuals today.
> In April 2009 RIPE accepted a policy proposal of January 2006 to assign IPv6 provider-independent IPv6 prefixes. Assignments are taken from the address range 2001:678::/29 and have a minimum size of a /48 prefix.
You can have your own PI bloc and move it between ISPs if you so desire. You effectively own the bloc.
I do a "light" version of this, but without running a public AS and using WireGuard for tunneling my public IPv4 subnet into my homelab (proxmox cluster).
Just running bird on my VPS to announce my routes to the upstream over a private link.
I looked into buying my own IP space from that IP auction site, an IPv4 C-class costs around $10,000. What stopped me was finding out I also to register with RIPE and pay the LIR annual fee, costing hundred Euros per month or so, even if I wasn't yet ready to use the IP space (I wanted to setup a basic Anycast IP without Cloudflare with help of VPS host who said they can help and had multiple locations around world).
While I strongly support IPv6 migration, the current IPv4 pricing is a rip-off. All the brokers and auction sites are fantasizing.
The market is tight, but nowhere near the point where it was 4-5 years ago. Big cloud providers already bought enormous amounts of IPv4 while many regional ISPs and colocation providers went out of business.
There is no real pressure to buy IPv4 except for brand-new companies to get their initial /24 or /23 to start. Everything else is optional.
When I bought my initial /24 on such a site, it was not a competitive auction. I was the only bidder, and I paid the opening bid price, which was set by the seller. It's true that it was a real price, in that I paid it, but the 'auction' aspect felt like a farce.
They keep details private. It's not something transparent like eBay or a public auction. I think it's just a scam to pressure buyers into offering more.
Back around 1993-94 was a genuine gold rush in terms of domain names and network numbers.
My supervisor one day rushed into the bullpen and proclaimed that he had registered SEX.ORG, and presumably the only reason was to squat it awhile and then resell it for thousands. [Squatting and speculation were, in fact, quite legal and wise moves at that point in history, especially with a high-demand 6-character site!]
Personally, I discovered the registration process and forms for domain names and network numbers were fairly straightforward. I had seen a Usenet post where someone explained that you just had to write a description of your company, its structure and annual meetings, finances, etc. So I completely made up a fictional company and described those things in my application.
Hey presto, I was now the "owner" and "admin" of cthulhu.com and a corresponding 192.0.0.0/3 Class-C network. Now my coworkers at the ISP were savvy enough to arrange for the DNS servers to answer for their vanity domains. But having no appreciable homelab, or BGP peering of my own, my DNS domain and Class-C Network both languished, until ultimately they were reclaimed in a sweep of unused space by IANA and InterNIC.
I have been unable to recall the exact numbers or find them in a search, but I know that its moniker was related, such as "CTHULHU-NET" or something.
I went on to legitimately register under the .ca.us domain on behalf of my home network and my roommates. cthulhu.com has long been handed over to someone who uses it.
I had named it "HEARTLAND" rather than a Cthulhu-related name, which was hindering my searches. I had also asked Gemini and it hallucinated a historical record which it was unable/unwilling to link.
My original assigned user handle was: RE229 (a prime number, very on-brand)
My Netcom email address and a San Jose phone number are enshrined in the record. Don't bother contacting me through those! Interestingly, if you spell out the phone number, it ends in "NET", but does not spell anything compelling in its entirety.
This is great. You still "own" it, as it still exists in "whois" and ARIN records! The problem is it is assigned to an email address you no longer have access to. You might need to contact ARIN to get back control of it... seems possible since it's in your name and not a company.
That is baffling. I swear that I heard or received a direct communiqué, many many years ago, that ARIN and ICANN and IANA were sweeping out all of the unannounced networks and reclaiming them. Word went out during the initial pressure of address space exhaustion.
So if this is really still assigned to "me" as boss of "my company" I suppose nobody else has ever announced it. It has no BGP behind it. In fact, 192.160.0.0/16 has no BGP at all. That is a huge swath of space to be vacant.
So, in 34 years since its registration, no BGP announcement, no ASN has ever been associated with this Class-C, and it still "belongs to me", unpaid, un-rented? It boggles my mind. I had expected that it was easier to lose an unused IPv4 network than to lose a domain name from back in the day.
Now there is a lot of crazy contact information that is so, so old. It is credible but I barely recall even having some of those phone numbers. There's an email address at cts.com. Which appears to be utterly defunct now but it was a San Diego bboard run by "Bill Blue". I distinctly recall a lot of Usenet posters using "crash.cts.com" and it was a "shell account" provider. It would've been in-character for past-me to sign up there at some point. Some point, I don't know.
So it says they last attempted to contact me 16 years ago. Did they send a letter in the USPS? Did my family receive nothing? So weird. If I literally wrote to them with that return address, would they validate me?
I literally have no idea how I could even use a /24 network today. My ISP wouldn't accept it. I can't exactly run BGP from a Chromebook or Netgear router at home! I suppose the only way to use it would be on a VPS service? Would the VPS announce an old-school "personal" network?
When ARIN contacts you, it's through email. They send an email with a confirmation link, so that probably bounced.
You can definitely announce BGP from a VPS with some providers. I have been doing this for years. Vultr will do it. However, they will validate (through email) that you "own" the block.
My recommendation is you first contact ARIN and see if you can "recover" contact info associated with the class C.
There are some restrictions around legacy blocks that predate the existing of ARIN. For some reason, they cannot reclaim them easily. So they just sit there...
If you have a ham radio licence (anywhere in the world) you can request a /24 if IPv4 space from AMPR for free.
It cannot be used commercially and should be in the ‘spirit’ of amateur radio. Unfortunately there’s also a bit of a backlog it seems (a couple of months) right now.
Quite a lot of interesting stuff - for example there are mesh networks setup worldwide that attempt to run IP over RF using these - and then use the internet to forward packets from one to another.
They also offer simpler ‘turn-key’ wireguard tunnels too for things like Web SDR setups.
For BGP direct announce in practice it seems to be in the spirt of non-commercial ‘self learning and experimentation’ which is what a lot of legislatures around the world do use as their base definition for the ‘amateur’ in amateur radio. So I guess much like having slices of radio frequencies reserved for it, we’re lucky there are slices of address space reserved for this.
You only need an LIR annual fee (~$2000) if you want to be an LIR and manage other people's resources. Otherwise you find another LIR (some popular choices are the ones the OP used) to manage your resources on your behalf. The annual fee is then ~$60. The resources are allocated directly to you, even when managed by a third party.
Yeah for single person use, this only really makes sense with IPv6. I'm interested in doing this in the near future and I think the yearly price for all-in (IPv6 /48 allocation, AS allocation + necessary VPS connections) comes out to about $200. It goes up to $300-400 if you want a PI subnet instead of PA (PI follows you to another LIR, PA does not).
If you do ever sign up with RIPE remember you can get a free /24 if it's the first one on your account. If you just buy one to start you've paid to lose that privilege.
> MSS clamping is non-negotiable with tunnels. Every layer of encapsulation eats into the MTU.
Can this tunnel be avoided somehow? If I have to choose between owning my prefix and having 1500 MTU, I'd probably take the latter: MTU issues are so annoying to deal with, and MSS-clamping doesn't solve all of them.
The whole point of BGP is to influence your routing tables. This fundamentally makes very little sense to do when you have a bunch of routers whose routing policy you don't control between you and whoever you're speaking BGP to. eBGP is just TCP and supports knobs to run over multiple hops (so up to 255), but at that point you can't really do anything with the routing information you exchange because the moment you hand the traffic off, the other party can do with it how it pleases. Also, very few people have enough public IP addresses for this, and on the Internet you obviously can't route RFC1918 space. Therefore, you need tunnels, so that you can be one hop away even if the tunneled traffic is traversing the Internet, and so that you can reach peers that let you announce whatever IP space you want.
The other thing you can do, of course, is to just do the same thing internal to your lab. You can absolutely stand up multiple ASN at home. I'd even argue that if you really want to learn BGP, this is a great way to do it, especially if you use two different platforms (say, FRR on FreeBSD peering with a cheap Mikrotik running RouterOS). That way you learn the underlying protocol and not a specific implementation, which is something that is very hard to undo in junior network engineers that have only ever been exposed to one way of doing things.
That's different from some of the goals outlined in the article, but if your goal is to learn this stuff rather than have provider-independent IP space (which even for home labs isn't very valuable to most people), doing it all yourself works fine.
You can use who you're physically connected to. If you have a physical or point–to–point connection to iFog and Lagrange Cloud, you don't need tunnels to reach them. Both these companies offer VPS services.
If your goal is to learn this stuff join dn42, the global networking lab, instead of wasting money with real allocations.
Yes, this can be avoided. All the standard advice and examples are tailored toward avoiding IP packet fragmentation entirely even when the tunnel transport can encapsulate and transmit packets larger than the underlying path MTU. Mostly this is justified for performance reasons, but it also tends to avoid even more difficult to debug situations where there's an MTU or ICMP issue between tunnel endpoints.
I haven't used Wireguard before, but I believe if you force the wg interface MTU to 1500, things will just work. I use IPSec where the solution would be to use something like link-layer tunneling that, ironically, adds another layer of encapsulation to the equation. Most tunnel solutions don't directly support fragmentation as part of their protocol, but you get it for free if they utilize, e.g., UDP or other disjoint IP protocol for transport and don't explicitly disable fragmentation (e.g. by requesting Don't Fragment (DF) flag).
If I were to do this (and I keep meaning to try), I might still lower the MSS on my server(s) just for performance reasons, but at least the tunnel would otherwise appear seamless externally.
If you want to be an LIR and have the right to manage other people's addresses on their behalf, as well as being a full member of the organisation with voting rights and so on. If you just need addresses, that's not you.
It's basically $275/year to have an AS and some PA assignment with no intermediary LIR. In Europe, you have to pay €1800/year without an ASN included. Each resource is billed separately. If you go with a middleman (another LIR) you usually have to pay 200€+ (with taxes) for 2 resources (ASN and PI space)
It's not comparable. You will lose your AS and PA if your sourcing-LIR goes out of business or increases prices against you. It's ab big difference to become a LIR or just a downstream customer.
You shouldn't lose an ASN or PI block, they are registered to you at RIPE, only managed by the LIR and can be transferred to another LIR in exceptional or routine circumstances. I think you'll have to pay another fee though.
A PA block is just part of a LIR's block that they give you permission to use, so I doubt you could keep that if they went out of business, but maybe RIPE has a procedure for it.
I don't want to blame anyone for using this setup except RIPE for their fee schedule. For example, I don't have IPv6 because that would double my running costs just for RIPE.
Ok, for reference, I'm paying my LIR about 150 EU/year. That includes the ASN, IPv6 /44 block, and a BGP VPS. I have a legacy IPv4 /24 block that doesn't cost me anything. I do miss out on RPKI for it because of that.
Outside of that, I have another couple of VPSes I use for experimenting with routing and fail over that are closer to my location. The VPSes are connected to my home network and each other over Wireguard.
€75 per assignment / sponsored resource
€50 per ASN assignment
So in your case it's only €50 for the AS-object but your LIR needs to allocate their LIR base fee costs (which includes the IPv6 PA that they assigned to you), also infrastructure, the VM, support.
I really like the couple of LIRs that are very active in supporting one-person-multihoming setups, but IMHO RIPE should directly do that at a low entry fee. RIPE has also education/training offerings for LIRs and that would increase the knowhow and reach of RIPE. Most home lab enthusiasts will apply the gained knowledge in their job, at customers and bring new customers to RIPE.
You're not wrong about the sustainability, but we don't know if it's their only business. I'm not sure how many BGP / multihome hobbyists there are out there, but I'm guessing it's not a huge number. My guess is the LIR stuff is an add on for their colocation and hosting services.
When I first made the ARIN or RIPE LIR decision a few years back, ARIN required a $500 registration fee for an ASN. That was a few years of LIR fees right there. I felt it was worth the risk for the savings with a hobbyist setup.
If you'd like to experiment with running your own AS in private address space, connecting to a friendly network of geeks over wireguard tunnels, check out DN42 https://dn42.dev/Home.
It's a great way to explore routing technologies and safely experiment with your own AS, running the same protocols as the "real" Internet, just in private space.
If you do get set up, give me a shout (https://markround.com/dn42), I'd be happy to peer with you if you want to expand beyond the big "autopeer" networks :)
This is really an amazing resource. If you don't know BGP and how to grok AS's, you aren't a fully actualized IP networking human.
This phrasing made me envision a future where I have 90% android replacement parts, and I actually need to know.
Not to nitpick, but the title should have AS capitalized. It’s confusing with the current capitalization.
Right! I was confused for a bit until I started reading it.
Otherwise, getting to know the power of FreeBSD is awesome. Thanks for creating the blog!
I think HN tends to undo all caps words unless it's an acronym HN specifically recognizes. Guessing BGP, GRE, and FreeBSD are understood but AS is not.
It’s too late now, but when submitting a post the poster has a window of time to edit the title. Useful for example when HN auto-edits to capitalisation get some words wrong. When you edit the title, those auto-edits are not applied to your edited title.
I would imagine AS is as common as the software you'd use to run one (FRR) right?
I was hoping with IPv6, getting an address space as an individual would go back to how it was in the early IPv4 days, but alas you need to be a multihomed individual with tons of usage instead of just a sophisticated netzien that wants to own their block.
One of my customers was handing out /64s for a while but it was more hassle than it was worth. I only ever saw one residential customer use it, and he was just smart enough to cause problems.
Its one of those things that there needs to be strong consumer demand for, or it will just never happen tbh.
From our perspective, what we want more than anything in the universe is to never do NAT or DNS ever again. I would much rather maintain a billing system indicating you rent a small block of IPV6 space, with a nice little static route, over maintaining never ending NAT and DNS logs for the benefit of police forces who cant shit without collecting every micron of data. But NAT is basically security these days, and theres a negative driver in exposing customer routers directly to the internet (in that, if it even supports v6 its likely to be rooted) Customers will leave if telcos do things properly, and theres literally zero reward for being nice about it.
Interesting, my two ISPs (one in Belgium, one in France, not business ISPs) hand out fixed /48 blocks to every customer. As far as I know, that's what RIPE recommends, they actively discourage from assigning longer prefixes than /56.
The modems they provide handle it without needing anything special from the customers. The devices get IPv6 addresses from this prefix, and are firewalled by default. It's pretty simple so I'm not sure what could go wrong there.
In some countries you're only required to turn over logs that you chose to collect, but you're not required to collect them.
Yes, same here. Very frustrating. It is almost as if the powers that be don't want lowly netizens controlling their own destiny.
Actually, they don't want to pollute the internet routing table with routes that are fully subsumed into other routes. The effect on address ownership is a side effect.
Actually, they just want to milk the money out of you. It's a matter of how much your willing to pay, as a business customer, it's all possible.
Most ISP do not have such pure goals, as to protect the global routing tables ;)
RIRs, not ISPs, allocate addresses at the top level, they make money on each address allocation, and they still won't allocate addresses to you if you don't multihome because they have a duty to conserve resources.
When you get PI addresses your LIR/ISP just passes your data on to the RIR.
Just like many industries there's a retail side and a wholesale side. You're asking to get a wholesale product from a retail channel. If you become a wholesale customer you can get what you want, for a price.
I don't want an address, they should be cheap, meaningless (sans routing, the longer the common prefix, the closer geographically you should be) and not conflated with identifiers.
I just want a way to do public-key based discovery. I'm not sure if wireguard + DHT would do though as it'd also mean that it's easy to track your PK (and maybe you through your devices/services announced with PKs).
Maybe you can announce your IP in a neat encryption scheme that adds some privacy without increasing costs too much?
Basically Yggdrasil?
Link: https://yggdrasil-network.github.io/
Oh, that's interesting
What is the point of owning public address space?
Anything in your private network (even if it goes over public internet) should be encrypted and locked up anyway. Something like Wireguard or Nebula only needs a few (maybe just one) publicly accessible address. Inside the overlay network, it's easy to keep IP addresses stable.
Anything public-facing likely needs a DNS record, updatable quickly when the IP of a publicly accessible interface changes (infrequently).
What am I missing?
The realistic point is to have your own abuse email contact, to evade the banhappy policies that most server hosts have even when you did nothing wrong. Usually they suspend your account if you don't reply within 24 hours, even if the complaint is obvious nonsense.
It's the only real way of running reliable IPv6 networks with multiple uplinks. Unless you want NATv6.
DNS updates are slow. BGP can react to a downed link in <1 sec.
Even fast LACP needs three seconds and that's on the same collision domain.
How does BGP actually detect a link is down? Keep alive default is 30s but that can be changed. If you set it to say one second, is that wise? Once a link is down, that fact will propagate at the speed of BGP and other routing protocols. Recovery will need a similar propagation.
Depending on where the link is, a second can be a "life time" these days or not. It really depends on the environment what an appropriate heart beat interval might be.
Also, given that BGP is TCP based, it might have to interact with other lower level link detection protocols.
BFD or Ethernet-OAM is the standard here.
It can get a bit hardware dependant but getting <50ms failovers from software based BFD in BIRD or FRR is fairly easy, and I've tested down to < 1ms before with hardware based BFD echo. ~50ms is the point at which a user making a traditional VOIP call won't notice the path switch.
You can get NIC's for computers (like most Nvidia/Meallanox or higher end Broadcom/Intel NIC's that do hardware BFD, and its obviously included in higher end networking kit.
You then link the BGP routes to the health of the BFD session for which that path is the next hop, and you get super quick withdrawls.
I.e. bird detects interface failure but this affects only your side of decision making. For bidirectional failure detection you do BFD with BGB. BFD default timers are 3 times 30 ms, iirc.
I have both my own multihomed ASN and operate my own nameservers. The latter has usually been about as fast for failover overall in practice. BGP may look to converge near instantly from your 2-3 peer outbound perspective but the inbound convergence from the 100k networks on the rest of the internet is much slower and has a long tail very akin to trying to set your DNS TTL to 0 and having the rest of the internet decide to do it slower for cache/churn reasons anyways.
The bigger problem, and where BGP multihoming is most handy, is it's just so much easier to get a holistic in+out failover where nothing really changes vs in DNS where it's more about getting the future inbound stuff to change where it goes. E.g. it's a pain to break an active session because the address had to change, even if DNS can update where the new service is quickly.
The long tail of routers receiving your update doesn’t matter. Once the common transit networks get it, that’s where the rest would dump the traffic to reach you anyway. The only time slow propagation to the edges matters is the first time announcing a prefix after it has been fully withdrawn.
Using the wrong route to get the packet in your general direction still gets you the packet as long as it hits an ISP along the way that got the update.
We could fully drain traffic from a transit provider in <60s with a withdrawal with all of the major providers you get at the internet exchanges. If you weren’t seeing that your upstream ISPs may have penalized you for flapping too much and put in explicit delays.
<60s sounds about right as a general safe estimate. I just mean people should expect 1-2ish orders of magnitude more than <1s from a downed link with internet BGP upstreams in a multihomed situation.
I’m saying that’s not a correctly configured link for fast failure.
<1 second was normal for hard link down events or explicit withdrawals. Anything above that was waiting for some BGP peer timeout or some IGP event.
If your ISP is taking longer than 1 second to propagate your change, you’ve been put in some dunce protection box.
If it were flap suppression/slow peer detection/"the dunce bucket" there wouldn't be a long tail of convergence - it'd just be nothing until all at once. This also isn't something I've only seen on my personal AS alone, it's what I've come to expect in many enterprise cutovers while previously working at a network VAR. The personal AS is however much more carefree to move around to different random providers on a whimthough of course :).
I found some data from an oldish post by benjojo https://blog.benjojo.co.uk/post/speed-of-bgp-network-propaga... which confirm various tirr 1s do propagate updates across their networks very fast (<2ish seconds) while others certainly do not. Notably, Level 3 (now Lumen) is the largest BGP presence by prefix count and was the worst tested in the list - starting to apply at ~20s after to finishing at ~50s after. This was for announce specifically, which should be the clearer case.
Honestly it's not free but it's really not that expensive. With RIPE it's about 75€ per year for the ASN and being multihomed is not really a problem, there are multiple services that will let you announce through them for free or very cheap. You don't have volume minimums.
I do agree it should be simpler, but it is accessible to individuals today.
I feel you. Us nerds have been ignored by modern day home user contracts.
Que? 4,722,366,482,869,645,213,696 addresses isn't enough for you?
They want the address block registered directly to them instead of their ISP
> In April 2009 RIPE accepted a policy proposal of January 2006 to assign IPv6 provider-independent IPv6 prefixes. Assignments are taken from the address range 2001:678::/29 and have a minimum size of a /48 prefix.
You can have your own PI bloc and move it between ISPs if you so desire. You effectively own the bloc.
Presentation How to become your own ISP at What Hackers Yarn camp 2025
Highly recommended for those interested
https://media.ccc.de/v/why2025-9-how-to-become-your-own-isp https://youtube.com/watch?v=raHBq0rUdJQ
`-rxcsum -txcsum -rxcsum6 -txcsum6 -lro -tso`
Why disable all offloading? It's not explained anywhere.
Poor driver support on the poster's particular hardware, maybe?
In that case they should add a warning there in my opinion. It makes a lot of difference in my testing
I do a "light" version of this, but without running a public AS and using WireGuard for tunneling my public IPv4 subnet into my homelab (proxmox cluster).
Just running bird on my VPS to announce my routes to the upstream over a private link.
I'd love to read about your setup!
I looked into buying my own IP space from that IP auction site, an IPv4 C-class costs around $10,000. What stopped me was finding out I also to register with RIPE and pay the LIR annual fee, costing hundred Euros per month or so, even if I wasn't yet ready to use the IP space (I wanted to setup a basic Anycast IP without Cloudflare with help of VPS host who said they can help and had multiple locations around world).
While I strongly support IPv6 migration, the current IPv4 pricing is a rip-off. All the brokers and auction sites are fantasizing.
The market is tight, but nowhere near the point where it was 4-5 years ago. Big cloud providers already bought enormous amounts of IPv4 while many regional ISPs and colocation providers went out of business.
There is no real pressure to buy IPv4 except for brand-new companies to get their initial /24 or /23 to start. Everything else is optional.
How can an auction site fantasize? The price is what someone bid, and that's the real price.
When I bought my initial /24 on such a site, it was not a competitive auction. I was the only bidder, and I paid the opening bid price, which was set by the seller. It's true that it was a real price, in that I paid it, but the 'auction' aspect felt like a farce.
They keep details private. It's not something transparent like eBay or a public auction. I think it's just a scam to pressure buyers into offering more.
Note that it is not a real C-class IP prefix unless it is from the 192.0.0.0/3 range, otherwise it is just a sparkling /24 IP prefix.
Back around 1993-94 was a genuine gold rush in terms of domain names and network numbers.
My supervisor one day rushed into the bullpen and proclaimed that he had registered SEX.ORG, and presumably the only reason was to squat it awhile and then resell it for thousands. [Squatting and speculation were, in fact, quite legal and wise moves at that point in history, especially with a high-demand 6-character site!]
Personally, I discovered the registration process and forms for domain names and network numbers were fairly straightforward. I had seen a Usenet post where someone explained that you just had to write a description of your company, its structure and annual meetings, finances, etc. So I completely made up a fictional company and described those things in my application.
Hey presto, I was now the "owner" and "admin" of cthulhu.com and a corresponding 192.0.0.0/3 Class-C network. Now my coworkers at the ISP were savvy enough to arrange for the DNS servers to answer for their vanity domains. But having no appreciable homelab, or BGP peering of my own, my DNS domain and Class-C Network both languished, until ultimately they were reclaimed in a sweep of unused space by IANA and InterNIC.
I have been unable to recall the exact numbers or find them in a search, but I know that its moniker was related, such as "CTHULHU-NET" or something.
I went on to legitimately register under the .ca.us domain on behalf of my home network and my roommates. cthulhu.com has long been handed over to someone who uses it.
I remember those days. Anyone could get a /24 if they filled out the form and emailed it to Internic.
I'm still holding my early 90's "class C" and have it routed to my home network. It is legacy space, I never signed the ARIN RSA, so it remains free.
I found it!
https://rscott.org/OldInternetFiles/network-contacts.1996061...
I had named it "HEARTLAND" rather than a Cthulhu-related name, which was hindering my searches. I had also asked Gemini and it hallucinated a historical record which it was unable/unwilling to link.
The network was: 192.160.182.0/24
ARIN still has the history: https://whois.arin.net/rest/ip/192.160.182.0?s=192.160.182.0
My original assigned user handle was: RE229 (a prime number, very on-brand)
My Netcom email address and a San Jose phone number are enshrined in the record. Don't bother contacting me through those! Interestingly, if you spell out the phone number, it ends in "NET", but does not spell anything compelling in its entirety.
This is great. You still "own" it, as it still exists in "whois" and ARIN records! The problem is it is assigned to an email address you no longer have access to. You might need to contact ARIN to get back control of it... seems possible since it's in your name and not a company.
That is baffling. I swear that I heard or received a direct communiqué, many many years ago, that ARIN and ICANN and IANA were sweeping out all of the unannounced networks and reclaiming them. Word went out during the initial pressure of address space exhaustion.
So if this is really still assigned to "me" as boss of "my company" I suppose nobody else has ever announced it. It has no BGP behind it. In fact, 192.160.0.0/16 has no BGP at all. That is a huge swath of space to be vacant.
So, in 34 years since its registration, no BGP announcement, no ASN has ever been associated with this Class-C, and it still "belongs to me", unpaid, un-rented? It boggles my mind. I had expected that it was easier to lose an unused IPv4 network than to lose a domain name from back in the day.
Now there is a lot of crazy contact information that is so, so old. It is credible but I barely recall even having some of those phone numbers. There's an email address at cts.com. Which appears to be utterly defunct now but it was a San Diego bboard run by "Bill Blue". I distinctly recall a lot of Usenet posters using "crash.cts.com" and it was a "shell account" provider. It would've been in-character for past-me to sign up there at some point. Some point, I don't know.
So it says they last attempted to contact me 16 years ago. Did they send a letter in the USPS? Did my family receive nothing? So weird. If I literally wrote to them with that return address, would they validate me?
I literally have no idea how I could even use a /24 network today. My ISP wouldn't accept it. I can't exactly run BGP from a Chromebook or Netgear router at home! I suppose the only way to use it would be on a VPS service? Would the VPS announce an old-school "personal" network?
You may have been looking in the wrong spot? You can see plenty of prefixes in 192.160.0.0/16 currently announced. Check out https://stat.ripe.net/widget/routing-history
When ARIN contacts you, it's through email. They send an email with a confirmation link, so that probably bounced.
You can definitely announce BGP from a VPS with some providers. I have been doing this for years. Vultr will do it. However, they will validate (through email) that you "own" the block.
My recommendation is you first contact ARIN and see if you can "recover" contact info associated with the class C.
There are some restrictions around legacy blocks that predate the existing of ARIN. For some reason, they cannot reclaim them easily. So they just sit there...
They use this site now instead:
https://cthulhuventures.com/
If you have a ham radio licence (anywhere in the world) you can request a /24 if IPv4 space from AMPR for free.
It cannot be used commercially and should be in the ‘spirit’ of amateur radio. Unfortunately there’s also a bit of a backlog it seems (a couple of months) right now.
Oh, interesting. What's at the intersection of networking and amateur radio that these address blocks are often used for?
Quite a lot of interesting stuff - for example there are mesh networks setup worldwide that attempt to run IP over RF using these - and then use the internet to forward packets from one to another.
They also offer simpler ‘turn-key’ wireguard tunnels too for things like Web SDR setups.
For BGP direct announce in practice it seems to be in the spirt of non-commercial ‘self learning and experimentation’ which is what a lot of legislatures around the world do use as their base definition for the ‘amateur’ in amateur radio. So I guess much like having slices of radio frequencies reserved for it, we’re lucky there are slices of address space reserved for this.
You only need an LIR annual fee (~$2000) if you want to be an LIR and manage other people's resources. Otherwise you find another LIR (some popular choices are the ones the OP used) to manage your resources on your behalf. The annual fee is then ~$60. The resources are allocated directly to you, even when managed by a third party.
Yeah for single person use, this only really makes sense with IPv6. I'm interested in doing this in the near future and I think the yearly price for all-in (IPv6 /48 allocation, AS allocation + necessary VPS connections) comes out to about $200. It goes up to $300-400 if you want a PI subnet instead of PA (PI follows you to another LIR, PA does not).
If you do ever sign up with RIPE remember you can get a free /24 if it's the first one on your account. If you just buy one to start you've paid to lose that privilege.
If you can register on ARIN the costs are only $260/year at the smallest tier and you can also apply for a /24 which you should be able to get.
iFog and Lagrange Cloud, naturally.
I am always very curious why these operations exist. ISPs for the very specific niche of hobbyists who want to run ASNs.
> MSS clamping is non-negotiable with tunnels. Every layer of encapsulation eats into the MTU.
Can this tunnel be avoided somehow? If I have to choose between owning my prefix and having 1500 MTU, I'd probably take the latter: MTU issues are so annoying to deal with, and MSS-clamping doesn't solve all of them.
Kind of but not really.
The whole point of BGP is to influence your routing tables. This fundamentally makes very little sense to do when you have a bunch of routers whose routing policy you don't control between you and whoever you're speaking BGP to. eBGP is just TCP and supports knobs to run over multiple hops (so up to 255), but at that point you can't really do anything with the routing information you exchange because the moment you hand the traffic off, the other party can do with it how it pleases. Also, very few people have enough public IP addresses for this, and on the Internet you obviously can't route RFC1918 space. Therefore, you need tunnels, so that you can be one hop away even if the tunneled traffic is traversing the Internet, and so that you can reach peers that let you announce whatever IP space you want.
The other thing you can do, of course, is to just do the same thing internal to your lab. You can absolutely stand up multiple ASN at home. I'd even argue that if you really want to learn BGP, this is a great way to do it, especially if you use two different platforms (say, FRR on FreeBSD peering with a cheap Mikrotik running RouterOS). That way you learn the underlying protocol and not a specific implementation, which is something that is very hard to undo in junior network engineers that have only ever been exposed to one way of doing things.
That's different from some of the goals outlined in the article, but if your goal is to learn this stuff rather than have provider-independent IP space (which even for home labs isn't very valuable to most people), doing it all yourself works fine.
You can use who you're physically connected to. If you have a physical or point–to–point connection to iFog and Lagrange Cloud, you don't need tunnels to reach them. Both these companies offer VPS services.
If your goal is to learn this stuff join dn42, the global networking lab, instead of wasting money with real allocations.
Yes, this can be avoided. All the standard advice and examples are tailored toward avoiding IP packet fragmentation entirely even when the tunnel transport can encapsulate and transmit packets larger than the underlying path MTU. Mostly this is justified for performance reasons, but it also tends to avoid even more difficult to debug situations where there's an MTU or ICMP issue between tunnel endpoints.
I haven't used Wireguard before, but I believe if you force the wg interface MTU to 1500, things will just work. I use IPSec where the solution would be to use something like link-layer tunneling that, ironically, adds another layer of encapsulation to the equation. Most tunnel solutions don't directly support fragmentation as part of their protocol, but you get it for free if they utilize, e.g., UDP or other disjoint IP protocol for transport and don't explicitly disable fragmentation (e.g. by requesting Don't Fragment (DF) flag).
If I were to do this (and I keep meaning to try), I might still lower the MSS on my server(s) just for performance reasons, but at least the tunnel would otherwise appear seamless externally.
Just a reminder, that the basic fees at RIPE are 2-3x the fees at ARIN which hurts individuals, SOHO and multihomed not-for-profit institutions.
fee schedules FYI
- ARIN 2026 PDF: https://www.arin.net/resources/fees/images/2026feeschedule.p...
- RIPE 2026 : https://www.ripe.net/membership/payment/
Enthusiasts, trainees and small orgs are paying a lot more with RIPE.
If you want to be an LIR and have the right to manage other people's addresses on their behalf, as well as being a full member of the organisation with voting rights and so on. If you just need addresses, that's not you.
Your ARIN link is broken.
fixed arin link: https://www.arin.net/resources/fees/fee_schedule/
It's basically $275/year to have an AS and some PA assignment with no intermediary LIR. In Europe, you have to pay €1800/year without an ASN included. Each resource is billed separately. If you go with a middleman (another LIR) you usually have to pay 200€+ (with taxes) for 2 resources (ASN and PI space)
> PA assignment with no intermediary LIR
No such thing. PA by definition is a slice of your LIR's address block.
But op claims to be independent by using PA space of their LIR…
Good to know. As someone on the ARIN side, I always found the fees reasonable.
You can get better deals with the right LIR. As a hobbyist it was cheaper for me to go with a RIPE LIR over ARIN.
See: https://lagrange.cloud/products/lir
It's not comparable. You will lose your AS and PA if your sourcing-LIR goes out of business or increases prices against you. It's ab big difference to become a LIR or just a downstream customer.
You shouldn't lose an ASN or PI block, they are registered to you at RIPE, only managed by the LIR and can be transferred to another LIR in exceptional or routine circumstances. I think you'll have to pay another fee though.
A PA block is just part of a LIR's block that they give you permission to use, so I doubt you could keep that if they went out of business, but maybe RIPE has a procedure for it.
I do not know anyone that have PI recently. It is exceptional to issue these days
For a hobbyist it’s perfectly fine, I think? I’ve been doing this for years. If I was a major corporation I might be more concerned.
I don't want to blame anyone for using this setup except RIPE for their fee schedule. For example, I don't have IPv6 because that would double my running costs just for RIPE.
Ok, for reference, I'm paying my LIR about 150 EU/year. That includes the ASN, IPv6 /44 block, and a BGP VPS. I have a legacy IPv4 /24 block that doesn't cost me anything. I do miss out on RPKI for it because of that.
Outside of that, I have another couple of VPSes I use for experimenting with routing and fail over that are closer to my location. The VPSes are connected to my home network and each other over Wireguard.
This is not sustainable IMHO. RIPE fees are:
€75 per assignment / sponsored resource €50 per ASN assignment
So in your case it's only €50 for the AS-object but your LIR needs to allocate their LIR base fee costs (which includes the IPv6 PA that they assigned to you), also infrastructure, the VM, support.
I really like the couple of LIRs that are very active in supporting one-person-multihoming setups, but IMHO RIPE should directly do that at a low entry fee. RIPE has also education/training offerings for LIRs and that would increase the knowhow and reach of RIPE. Most home lab enthusiasts will apply the gained knowledge in their job, at customers and bring new customers to RIPE.
You're not wrong about the sustainability, but we don't know if it's their only business. I'm not sure how many BGP / multihome hobbyists there are out there, but I'm guessing it's not a huge number. My guess is the LIR stuff is an add on for their colocation and hosting services.
When I first made the ARIN or RIPE LIR decision a few years back, ARIN required a $500 registration fee for an ASN. That was a few years of LIR fees right there. I felt it was worth the risk for the savings with a hobbyist setup.
how much does it cost?