Follow

What's the easiest way to set up a simulated network with ISP-style NAT? Preferably without buying dedicated hardware for the purpose, and preferably reproducible for other developers.

Usecase: I am developing a P2P system, and need a reasonably representative environment to test my software in, and how well it deals with shitty residential networking configurations.

I'm a developer, not a network engineer, so my knowledge of networks is limited to a developer perspective and I don't have the spoons to learn it in-depth.

Boosts appreciated :boost_requested:

@joepie91 "ISP-style NAT" is a very wide range of weird varieties (varying as well by country/company), so it's fairly hard to be sure exactly what you're looking for.
That said, you might be looking for something along the lines of having multiple routers' WAN ports connected up to a single parent router, subnetted to be completely independent of each other but ultimately able to connect over the Internet.
I'm curious exactly what your P2P system is and why you'd need to test this specifically

@paarth @joepie91 gotta also test the scenarios for UPnP working, UPnP not working and UPnP pretending to work but actually doing something ridiculous (I've only dealt with it directly once 5+ years ago but I remember there were many surprises).

@joepie91 set up a machine behind a consumer router and disable all port mapping protocols

@joepie91 This is not an answer, but I really hope there *is* a good answer.

I'd love to have an "internet" VM that simulates a (small scale) global internet, with packet losses, variable latency, etc., in a configurable way, along with a way to hook up individual client VMs to the fake-internet (with simulated NAT etc. like you mention). A turn-key solution really ought to be possible, but I'm not sure one exists.
@joepie91 Seems like there was a GSoC project to add NAT simulation support to ns-3, but it was either never finished or never merged upstream https://groups.google.com/g/ns-3-users/c/UnzXZqOUWfk

https://www.nsnam.org/about/

@retr0id @joepie91 Yes, that'd be really cool. This kind of stuff does exists in the IC development world where the physics and decay can be simulated, so you're not just running the HDL on an FPGA. Kind of weird the networking world doesn't do this.

@joepie91 should be possible using Linux network namespaces, they basically give you network-isolated environments on the same machine, without having to run complete VMs.

@joepie91 I am in the same situation as you and I hate to say it but I have come to the conclusion that there's no substitute for the real thing.

In terms of simulating just the NAT part, the default networking configurations of Libvirt VMs and Docker containers should get you there. So for example, you could test two containers which are on different docker "bridged" networks, and see if you can get them to talk P2P in the usual way: the two will rendezvous somewhere like a STUN server, exchange each-others source ports, and then attempt to connect to each-other via UDP.

So for example:

* container A decides "I will create a socket with source port 12345"
* container B decides "I will create a socket with source port 43211"
* they exchange that info with eachother
* container A opens a UDP socket to the docker bridge IP address with source port 12345 and destination port 43211
* container B opens a UDP socket to the docker bridge IP address with source port 43211 and destination port 12345

To be honest I've never tried this, I have no idea if it will work but based on my somewhat limited understanding, it _should_ work for initial prototyping.

However the problem I see is that when it comes time to polish the application, you will run into situations where every home router is slightly different. I'm not aware of any tools which can simulate that. So far I have simply tried to deploy apps in as many different home networks as possible, those of my friends and family.

@joepie91 at previous job we turned up ESXi in a VM, and ran quite a lot of DNS servers and other stuff to simulate a network

Today I would run the same in Kubernetes or dockers

There is nothing /that/ special about an ISP network

Sign in to participate in the conversation
Pixietown

Small server part of the pixie.town infrastructure. Registration is closed.