As you predicted, it doesn’t matter much if you keep your router and modem together — at least when it comes to getting internet access to your office. Personally, I’d keep them nearby for troubleshooting, like you mentioned. Bringing your router up from the basement could help wireless reception in other areas of the house, though. It might be worth moving it up to your new office temporarily to see if it helps internet access in far-off rooms.
When it comes to coaxial types, RG-6 is best suited for high-bandwidth tasks like TV signals and broadband internet. RG-59 is a less capable cable typically reserved for things like closed-circuit security systems. Cat7 Ethernet is your best bet if you want to completely future-proof your setup, since it supports speeds up to 10Gb/s. Cat6 is cheaper, though, and can get those speeds in short runs.
It’s a good thing you’ve already committed to stringing cables, because otherwise you’d have to look at things like powerline internet to spread network access through your home. I’ve heard good things about those systems, but they also involve buying entirely new equipment. As usual, it’s much easier just to throw a cable into a new room. Wi-Fi is also getting better, but to get the best speeds you’d need to use a 5GHz network, which doesn’t pass through walls and floors easily. A multi-room system like Eero and Google WiFi could help too, but they still cost a lot, and technically won’t be as reliable as a physical cable.