One challenge I have found is that due to the entries which have to be loaded into dnsmasq by pi-hole, that adversely affects dnsmasq's ability to cache real DNS entries. The workaround I have for this is to use 2 Raspberry Pis - one running Pi-hole and a second one running dnsmasq (with a cache of 10000 entries as well) as my DNS and DHCP server.
While that means that DNS queries for "real-world" DNS records go through two steps on my network, I am still getting responses to cached entries in 2ms as opposed to about 170ms when DNS queries are forwarded externally. In addition, I am finding that my DNS cache is now about 95% effective - roughly 4.5% of queries that hit the dnsmasq instance that is acting as my DNS/DHCP server get forwarded. Once the blocked DNS queries and the cache hits on the pi-hole are added, that will give a few more percentage points overall. I haven't done the calculations yet, but I would guess that about 96% to 97% of all my DNS queries are being answered locally.
Caching is still working, but it only appears to be about 25% effective. I wonder what would happen if dnsmasq could run with a cache of over 100000 entries? That may be what is needed to get a good caching rate working again.
Oh Yes, I forgot about that. I'm using a patched version of dnsmasq which I compiled from the source code. One of my changes actually was to remove this arbitrarily set limit of 10'000 and I'm using 250'000 in my config file (again an arbitrary but notably higher number). I forgot about that since this patch is older than one year and re-applied when I compile a more recent version of dnsmasq.
You may want to get into contact with the author of dnsmasq (Simon Kelley, simon@thekelleys.org.uk) and ask him whether he can remove this limit in the upstream version of dnsmasq.
Have you verified the your cache hit rate after doing that? The reason I ask is that even with a cache size of 250000 entries I am still only seeing about 25% hit rate on dnsmasq's cache on the pi-hole. Realistically, I would be expecting a hit rate of over 90% - which is what I see on my other dnsmasq instance.
$ grep 'google.de' /var/log/pihole.log
Jun 19 17:48:42 dnsmasq[15090]: query[A] google.de from 127.0.0.1
Jun 19 17:48:42 dnsmasq[15090]: forwarded google.de to 2620:0:ccd::2
Jun 19 17:48:42 dnsmasq[15090]: reply google.de is 172.217.17.163
Jun 19 17:48:45 dnsmasq[15090]: query[A] google.de from 127.0.0.1
Jun 19 17:48:45 dnsmasq[15090]: cached google.de is 172.217.17.163
Jun 19 17:49:48 dnsmasq[15090]: query[A] google.de from 127.0.0.1
Jun 19 17:49:48 dnsmasq[15090]: cached google.de is 172.217.17.163
Jun 19 17:51:50 dnsmasq[15090]: query[A] google.de from 127.0.0.1
Jun 19 17:51:50 dnsmasq[15090]: cached google.de is 172.217.17.163
Jun 19 17:55:50 dnsmasq[15090]: query[A] google.de from 127.0.0.1
Jun 19 17:55:50 dnsmasq[15090]: cached google.de is 172.217.17.163
look for line 2521 in option.c (the line number may be slightly different in my version of this file):
if (size < 0)
size = 0;
// else if (size > 10000)
// size = 10000;
daemon->cachesize = size;
As you can see, the only thing I did was removing the upper limitation on the cache size. Hence, I'm able to specify any cache size in my dnsmasq config file.
any chance someone can link to or quickly type up a guide to running a second dnsmasq instance locally or configuring dnsmasq on a separate device just for local caching?
Are there any negatives (outside of unblocking something and having the persistent referral to the blocking address) of compiling dnsmasq from source and making the limit 500k or a million entries? @DL6ER can you share some of your info?
Why would you do that if you can just do it with the local instance without having to run a second one? Just so that the second instance is not loading the lists?
If so, I doubt running on the same device is possible without major customization: you either have to create a virtual interface with another IP and bind a second daemon to this interface (which I don't know if that is even possible) or you'd have to encapsulate another dnsmasq in a container like docker.
On a second device, this will be much easier, but - of course - there will be the overhead of running a second device for no obvious reason (as you could simply do what I posted above and remove the caching limits in the source code). In this case, you would configure Pi-hole to use solely the IP of your secondary device as upstream server and on the seconds device, you would configure Pi-hole as usual, but you'd disable the blocking. This should be the easiest way of setting it up with a second machine.
I don't understand the "persistent referral to the blocking address" issue. This will always happen regardless if you use either the package or the compiled version. I'm running Pi-hole with the cache size unlimited by the code (and manually configured to 500,000) since more than a year without having encountered any problems at all.
As you can see, my cache utilization is about ~60% but I wouldn't expect a higher value due to two reasons:
I'm a rather conservative user, i.e. I go somewhere, do something and that's it. I don't browse around or stay a long time on the same webpage for which caching yould be favorable
I use only browsers with inbuilt DNS caching (Vivaldi), i.e. there is an intrinsic reduction of DNS queries that is sent out at all. If you are using browsers that aren't caching the requests, the percentage can easily be higher than 95%, I guess...
I'm sorry, I think I misunderstood the caching portion of dnsmasq. I thought the cache limit included everything listed under addn hosts, but it does not, correct?