Dnsmasq and its cache

On thought on using Pi-hole as your local DNS:

One challenge I have found is that due to the entries which have to be loaded into dnsmasq by pi-hole, that adversely affects dnsmasq's ability to cache real DNS entries. The workaround I have for this is to use 2 Raspberry Pis - one running Pi-hole and a second one running dnsmasq (with a cache of 10000 entries as well) as my DNS and DHCP server.
While that means that DNS queries for "real-world" DNS records go through two steps on my network, I am still getting responses to cached entries in 2ms as opposed to about 170ms when DNS queries are forwarded externally. In addition, I am finding that my DNS cache is now about 95% effective - roughly 4.5% of queries that hit the dnsmasq instance that is acting as my DNS/DHCP server get forwarded. Once the blocked DNS queries and the cache hits on the pi-hole are added, that will give a few more percentage points overall. I haven't done the calculations yet, but I would guess that about 96% to 97% of all my DNS queries are being answered locally.

Does that mean you experience that caching is not working at all for you when you use only one instance of dnsmasq?

Caching is still working, but it only appears to be about 25% effective. I wonder what would happen if dnsmasq could run with a cache of over 100000 entries? That may be what is needed to get a good caching rate working again.

dnsmasq has a hardcoded limit of 10000 entries in the cache, which we max out.

Oh Yes, I forgot about that. I'm using a patched version of dnsmasq which I compiled from the source code. One of my changes actually was to remove this arbitrarily set limit of 10'000 and I'm using 250'000 in my config file (again an arbitrary but notably higher number). I forgot about that since this patch is older than one year and re-applied when I compile a more recent version of dnsmasq.

You may want to get into contact with the author of dnsmasq (Simon Kelley, simon@thekelleys.org.uk) and ask him whether he can remove this limit in the upstream version of dnsmasq.

1 Like

As a first step, I was planning on doing exactly the same thing - just raising the maximum cache size - 250000 sounds like a reasonable number.

See also

http://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/2007q4/001647.html

Seems like Simon is willing to change it, maybe he just needs to be reminded :wink:

I just removed the max limitation in the source code, so I'm able to change it without recompilation.

Have you verified the your cache hit rate after doing that? The reason I ask is that even with a cache size of 250000 entries I am still only seeing about 25% hit rate on dnsmasq's cache on the pi-hole. Realistically, I would be expecting a hit rate of over 90% - which is what I see on my other dnsmasq instance.

For me, caching seems to be working fine, e.g.

$ grep 'google.de' /var/log/pihole.log
Jun 19 17:48:42 dnsmasq[15090]: query[A] google.de from 127.0.0.1
Jun 19 17:48:42 dnsmasq[15090]: forwarded google.de to 2620:0:ccd::2
Jun 19 17:48:42 dnsmasq[15090]: reply google.de is 172.217.17.163
Jun 19 17:48:45 dnsmasq[15090]: query[A] google.de from 127.0.0.1
Jun 19 17:48:45 dnsmasq[15090]: cached google.de is 172.217.17.163
Jun 19 17:49:48 dnsmasq[15090]: query[A] google.de from 127.0.0.1
Jun 19 17:49:48 dnsmasq[15090]: cached google.de is 172.217.17.163
Jun 19 17:51:50 dnsmasq[15090]: query[A] google.de from 127.0.0.1
Jun 19 17:51:50 dnsmasq[15090]: cached google.de is 172.217.17.163
Jun 19 17:55:50 dnsmasq[15090]: query[A] google.de from 127.0.0.1
Jun 19 17:55:50 dnsmasq[15090]: cached google.de is 172.217.17.163

In absolute numbers:

queries forwarded 5121, queries answered locally 10055

but I have not been very active today on the Internet (mostly have done 'offline' work).

Have you checked if your removal of the cache limitation is working?

$ grep 'cachesize' /var/log/pihole.log
Jun 18 14:15:37 dnsmasq[5738]: started, version 2.76 cachesize 500000

I seemingly chose even 500'000.

I'm regularly compiling / using the test versions of dnsmasq, but can't find the hard coded cache limitation setting in config.h.

Would you be so kind to indicate what it is you changed, to remove the hard coded cache size.

tx

It's inside option.c

I agree

look for line 2521 in option.c (the line number may be slightly different in my version of this file):

            if (size < 0)
              size = 0; 
//          else if (size > 10000)
//            size = 10000;
            
            daemon->cachesize = size;

As you can see, the only thing I did was removing the upper limitation on the cache size. Hence, I'm able to specify any cache size in my dnsmasq config file.

Any quick guide to build dnsmasq from source? :wink:

This is my personal copy. The compile commands are there as well. Install dnsmasq beforehand so that it adds the necessary services.

@spacemonkey Thanks, it still not like the cahce-size=250000

I'm sorry, I didn't understand what you mean.

any chance someone can link to or quickly type up a guide to running a second dnsmasq instance locally or configuring dnsmasq on a separate device just for local caching?

Are there any negatives (outside of unblocking something and having the persistent referral to the blocking address) of compiling dnsmasq from source and making the limit 500k or a million entries? @DL6ER can you share some of your info?

Thank you

Why would you do that if you can just do it with the local instance without having to run a second one? Just so that the second instance is not loading the lists?

If so, I doubt running on the same device is possible without major customization: you either have to create a virtual interface with another IP and bind a second daemon to this interface (which I don't know if that is even possible) or you'd have to encapsulate another dnsmasq in a container like docker.

On a second device, this will be much easier, but - of course - there will be the overhead of running a second device for no obvious reason (as you could simply do what I posted above and remove the caching limits in the source code). In this case, you would configure Pi-hole to use solely the IP of your secondary device as upstream server and on the seconds device, you would configure Pi-hole as usual, but you'd disable the blocking. This should be the easiest way of setting it up with a second machine.

I don't understand the "persistent referral to the blocking address" issue. This will always happen regardless if you use either the package or the compiled version. I'm running Pi-hole with the cache size unlimited by the code (and manually configured to 500,000) since more than a year without having encountered any problems at all.


As you can see, my cache utilization is about ~60% but I wouldn't expect a higher value due to two reasons:

  • I'm a rather conservative user, i.e. I go somewhere, do something and that's it. I don't browse around or stay a long time on the same webpage for which caching yould be favorable
  • I use only browsers with inbuilt DNS caching (Vivaldi), i.e. there is an intrinsic reduction of DNS queries that is sent out at all. If you are using browsers that aren't caching the requests, the percentage can easily be higher than 95%, I guess...

I'm sorry, I think I misunderstood the caching portion of dnsmasq. I thought the cache limit included everything listed under addn hosts, but it does not, correct?