Mdthbs

I'm wondering if there is a way to have libvirt push virtual network DNS information into the NetworkManager instance of dnsmasq. As always I'm working with RHEL 7 (7.6)

Background:
My team works with two different physical architectures that run a similar software suite. I am working on a way to virtualize the entire environment to make testing and development cheaper and more available to developers. As much as I would prefer to use containers, the operating system gets a more thorough workout and test by using virtual machines. The two environments differ in number of network segments, and naming conventions. IP addressing is similar.

The road so far:
I've configured NetworkManager to use dnsmasq by adding a config file to /etc/NetworkManager/config.d/00-use-dnsmasq.conf

# /etc/NetworkManager/conf.d/00-use-dnsmasq.conf

#

# This enables the dnsmasq plugin.

[main]

dns=dnsmasq

This launches dnsmasq controlled by NetworkManager and sets the nameserver in /etc/resolv.conf to 127.0.0.1. The upstream domain nameservers provided by dhcp are configured as forwarders for all zones in dnsmasq.

Enter my virtual networks:
Each network has two storage devices that also provide their own statically configured DNS for the rest of the real system.
Since we are virtualizing, any requests my hypervisor makes for each named domain should be forwarded to the respective DNS.

I built the following files (names & addresses changed to protect proprietary information) to define virtual networks in libvirt:

cake.xml

<network ipv6="yes">

    <name>cake</name>

    <bridge name="cake" stp="off"/>

    <ip address="172.16.0.1" netmask="255.255.255.0"/>

    <domain name="cake.local" localOnly="yes"/>

    <dns>

        <forwarder addr="172.16.0.41"/>

        <forwarder addr="172.16.0.42"/>

        <host ip="172.16.0.41">

            <hostname>plex1.cake.local</hostname>

            <hostname>plex1</hostname>

        </host>

        <host ip="172.16.0.42">

            <hostname>c-plex2.cake.local</hostname>

            <hostname>c-plex2</hostname>

        </host>

    </dns>

</network>

pie.xml

<network ipv6="yes">

    <name>pie</name>

    <bridge name="pie" stp="off"/>

    <ip address="172.16.1.1" netmask="255.255.255.0"/>

    <domain name="pie.local" localOnly="yes"/>

    <dns>

        <forwarder addr="172.16.1.41"/>

        <forwarder addr="172.16.1.42"/>

        <host ip="172.16.1.41">

            <hostname>p-plex1.pie.local</hostname>

            <hostname>p-plex1</hostname>

        </host>

        <host ip="172.16.1.42">

            <hostname>p-plex2.pie.local</hostname>

            <hostname>p-plex2</hostname>

        </host>

    </dns>

</network>

I then perform a virsh net-define and a virsh net-start for both networks:

virsh net-define cake.xml

virsh net-define pie.xml



virsh net-start cake

virsh net-start pie

At this point I can dig both sets of storage devices and get a response for their IP addresses from their respective running instance of dnsmasq.
dig p-plex2.pie.local @172.16.1.1 or dig c-plex2.cake.local @172.16.0.1

After starting the storage devices I can resolve any host for which each storage device has in its static DNS using the dnsmasq server.

I can then tear all this down, and bring the other virtual environment up.

tart.xml

<network ipv6="yes">

    <name>tart</name>

    <bridge name="tart" stp="off"/>

    <ip address="172.16.0.1" netmask="255.255.255.0"/>

    <domain name="tart.local" localOnly="yes"/>

    <dns>

        <forwarder addr="172.16.0.41"/>

        <forwarder addr="172.16.0.42"/>

        <host ip="172.16.0.41">

            <hostname>t-plex1.pie.local</hostname>

            <hostname>t-plex1</hostname>

        </host>

        <host ip="172.16.0.42">

            <hostname>t-plex2.pie.local</hostname>

            <hostname>t-plex2</hostname>

        </host>

    </dns>

</network>

Again running

virsh net-define tart.xml

virsh net-start tart

dig t-plex2.tart.local @172.16.0.1

This is where things get cumbersome. All the answers and hints for dnsmasq in these type of configurations say to add files to /etc/NetworkManager/dnsmasq.d

# /etc/NetworkManager/dnsmasq.d/01-cake.conf

#

# This file directs dnsmasq to forward any request to resolve

# names under the cake.local domain to 172.16.0.1

# this is dnsmasq configured for the cake network segment

server=/cake.local/172.16.0.1



# /etc/NetworkManager/dnsmasq.d/02-pie.conf

#

# This file directs dnsmasq to forward any request to resolve

# names under the pie.local domain to 172.16.1.1

# this is dnsmasq configured for the pie network segment

server=/cake.local/172.16.1.1



# /etc/NetworkManager/dnsmasq.d/01-tart.conf

#

# This file directs dnsmasq to forward any request to resolve

# names under the cake.local domain to 172.16.0.1

# this is dnsmasq configured for the tart network segment

server=/tart.local/172.16.0.1

The issue is when I switch from the cake & pie networks to tart, there is an ip collision between cake and tart. In addition I don't want cake & pie information in dnsmasq when running tart, the opposite is also true.

Is there a configuration I am missing that will add and remove these dnsmasq forwarders in the NetworkManager owned dnsmasq instance?
The goals are as follows:

• Allow system use as normal for regular dns requests for
  authentication, web browsing etc.


• Rapidly switch between cake + pie and tart virtual network configurations


• No need to change the filesystem: /etc/NetworkManager/dnsmasq.d/
  


• No cross configuration of tart vs cake & pie


• Minimize NetworkManager restarts