Rerouting public IP network traffic to different site
We have a deployment of servers in a data center in New York (at 60 Hudson Street, a telecom hotel) where we announce IPv4 and IPv6 prefixes via BGP. Most consumer, and even business, network connections, e.g. Verizon FiOS, Comcast/Spectrum cable Internet, do not offer BGP interconnect. All you get is the IP address that the ISP supplies to you, and sometimes you don’t even get a publicly routable IP address (i.e. it is behind NAT, and you can only initiate outbound connections, not receive inbound network connections). It can be useful to set up publicly routable IP addresses at arbitrary sites and locations.
We have another site in Westchester, New York, that has a high speed fiber connection, but the ISP does not offer BGP interconnect. The way we overcame this limitation so that we could route IP traffic to the Westchester site was to create a VPN tunnel (use OpenVPN or WireGuard or other favorite VPN) between the data center and the Westchester site. We set up OSPF to create an encrypted overlay network with RFC 1918 IP addresses. That OSPF configuration was for more than just two sites. Then we set up a GRE tunnel between the data center and Westchester. We then set up a few static routing rules on both sides, and now servers hosted in Westchester (which has much cheaper hosting) can communicate with the Internet via publicly routable IP addresses, theoretically as many as we want, subject to the limits that the data center connection (or our supply from ARIN) permit. Technically we could do it without the GRE tunnel, but the benefit of using a GRE tunnel is that there is automatic failover in a simple way. We could also do it without GRE just with some OSPF configuration, but that would take some further research to figure out how (for a later day).