Cloudflare 1001 DNS Errors

paul78 · December 2018

Am I the only one that runs into Cloudflare 1001 DNS errors every 2 to 3 clicks on the new TE? Just wondering if it's just me. It looked like the new TE is hosted at infosec.vanillacommunities.com and there's some DNS round-robin configured. But it looks correct to me.

Seems to be related to use of Tor but Cloudflare supports Tor and I don't have problems accessing any other Cloudflare protected systems with Tor.

paul78 · December 2018

BTW - in case anyone is looking into this problem - my only observation so far when this problem occurs on TE is when the source IP is IPv6.

shochan · December 2018

Didn't Cloudflare implement DNSSEC recently? Not sure if its related, but that would only be an assumption...I would definitely get with their support about it.

https://www.bleepingcomputer.com/news/security/cloudflare-makes-dnssec-activation-easy/

Meggo · December 2018

Thanks for raising this. I sent in a ticket this morning.

paul78 · December 2018

shochan said:

Didn't Cloudflare implement DNSSEC recently? Not sure if its related, but that would only be an assumption...I would definitely get with their support about it.

That's pretty cool to see. I would be surprised if it's related but you never know. I did just check vanillacommunities.com and I see that DNSSEC is configured and it looks to be configured correctly. But I see that infosecinstitute.com doesn't have DNSSEC configured.

@Meggo - Thanks. I'm curious to know what causes the issue if that's able to be disclosed.

Meggo · December 2018

@paul78 Next time this happens, mind grabbing a screen shot that includes the URL?

paul78 · December 2018

@Meggo - happens every 3-4 HTTPS request so it's not tough to reproduce. The URL is an https://community.infosecinstitute.com URL so it can happen on any forum page. The error is being served up by Cloudflare. Instead of a screen shot, I am enclosing the actual HTML source. Note that the source IP is IPv6.

An example screenshot is on the Cloudflare support site here - https://support.cloudflare.com/hc/en-us/articles/204165588-Error-1001-DNS-resolution-error

Actual error below:

<!DOCTYPE html>
<!--[if lt IE 7]> <html class="no-js ie6 oldie" lang="en-US"> <![endif]-->
<!--[if IE 7]>    <html class="no-js ie7 oldie" lang="en-US"> <![endif]-->
<!--[if IE 8]>    <html class="no-js ie8 oldie" lang="en-US"> <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en-US"> <!--<![endif]-->
<head>
<title>DNS resolution error | community.infosecinstitute.com | Cloudflare</title>
<meta charset="UTF-8" />
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta http-equiv="X-UA-Compatible" content="IE=Edge,chrome=1" />
<meta name="robots" content="noindex, nofollow" />
<meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=1" />
<link rel="stylesheet" id="cf_styles-css" href="/cdn-cgi/styles/cf.errors.css" type="text/css" media="screen,projection" />
<!--[if lt IE 9]><link rel="stylesheet" id='cf_styles-ie-css' href="/cdn-cgi/styles/cf.errors.ie.css" type="text/css" media="screen,projection" /><![endif]-->
<style type="text/css">body{margin:0;padding:0}</style>


<!--[if gte IE 10]><!--><script type="text/javascript" src="/cdn-cgi/scripts/zepto.min.js"></script><!--<![endif]-->
<!--[if gte IE 10]><!--><script type="text/javascript" src="/cdn-cgi/scripts/cf.common.js"></script><!--<![endif]-->



</head>
<body>
  <div id="cf-wrapper">
    <div class="cf-alert cf-alert-error cf-cookie-error" id="cookie-alert" data-translate="enable_cookies">Please enable cookies.</div>
    <div id="cf-error-details" class="cf-error-details-wrapper">
      <div class="cf-wrapper cf-header cf-error-overview">
        <h1>
          <span class="cf-error-type" data-translate="error">Error</span>
          <span class="cf-error-code">1001</span>
          <small class="heading-ray-id">Ray ID: 4853f8e26ca86bf8 &bull; 2018-12-07 03:40:40 UTC</small>
        </h1>
        <h2 class="cf-subheadline">DNS resolution error</h2>
      </div><!-- /.header -->

      <section></section><!-- spacer -->

      <div class="cf-section cf-wrapper">
        <div class="cf-columns two">
          <div class="cf-column">
            <h2 data-translate="what_happened">What happened?</h2>
            <p>You've requested a page on a website (community.infosecinstitute.com) that is on the <a data-orig-proto="https" data-orig-ref="www.cloudflare.com/5xx-error-landing?utm_source=error_100x" target="_blank">Cloudflare</a> network. Cloudflare is currently unable to resolve your requested domain (community.infosecinstitute.com). There are two potential causes of this:</p>
            <ul>
               <li><strong>Most likely:</strong> if the owner just signed up for Cloudflare it can take a few minutes for the website's information to be distributed to our global network.</li>
               <li><strong>Less likely:</strong> something is wrong with this site's configuration. Usually this happens when accounts have been signed up with a partner organization (e.g., a hosting provider) and the provider's DNS fails.</li>
            </ul>
          </div>

          
        </div>
      </div><!-- /.section -->

      <div class="cf-error-footer cf-wrapper">
  <p>
    <span class="cf-footer-item">Cloudflare Ray ID: <strong>4853f8e26ca86bf8</strong></span>
    <span class="cf-footer-separator">&bull;</span>
    <span class="cf-footer-item"><span>Your IP</span>: 2405:8100:8000:5ca1::e:7f51</span>
    <span class="cf-footer-separator">&bull;</span>
    <span class="cf-footer-item"><span>Performance &amp; security by</span> <a href="https://www.cloudflare.com/5xx-error-landing?utm_source=error_footer" id="brand_link" target="_blank">Cloudflare</a></span>
    
  </p>
</div><!-- /.error-footer -->


    </div><!-- /#cf-error-details -->
  </div><!-- /#cf-wrapper -->

  <script type="text/javascript">
  window._cf_translation = {};
  
  
</script>

</body>
</html>

JDMurray · December 2018

How are to connecting to the Internet using only IPv6? Through a proxy at your work?

paul78 · December 2018

JDMurray said:

How are to connecting to the Internet using only IPv6? Through a proxy at your work?

No - just via regular behind a NAT internet access. It's regular TOR browser but when the error asserts, I notice it's always an IPv6 address. I haven't encountered this problem with other Cloudflare protected sites.

I'd be curious if anyone else sees this problem with the latest TOR browser.

JDMurray · December 2018

I just accessed TE using Mozilla Brave (Version 0.57.18) and in a private window with TOR. I'm not getting any errors by just browsing around TE and not logged in.

If you are not using the Brave browser, give it a try with the TOR private window feature. If you are using Brave, log out of TE and see if the CF errors occur when you are not logged in.

shochan · December 2018

I was using Brave, but not the Tor within it...I just tried it, after a ton of Captcha verifying it finally loaded up TE...of course much slower load as expected.

JDMurray · December 2018

I didn't have too much of an issue with speed, but those initial captchas are freekin' annoying! When I was doing the Cloudflare captchas, it displayed my TOR exit IP as IPv4. Maybe the CF errors are related to IPv6.

paul78 · December 2018

Thanks guys. That's an interesting tidbit. I'm using Tor Browser from Tor Project. Cloudflare does support Tor exit nodes pretty well so perhaps it's something else. I'll see if I can play around with other settings.

paul78 · December 2018

@JDMurray and @shochan - thanks for your comments. I was just re-reading your posts and I noticed your mention about the captchas. Brave must be not have implemented opportunistic onions. I use TorBrowser 8.x so I don't see the capthas with Cloudflare protected sites. You rarely see captchas with Cloudflare protected sites because, CF created an Onion service and it looks like it's enabled for TE.

There's a pretty interesting blog article on the topic here - https://blog.cloudflare.com/cloudflare-onion-service/

Perhaps the problem is related to TE through an opportunistic onion. I noticed some patches to TorBrowser this week so I'll check those out.

Meggo · December 2018

Let me know if you dig up anything interesting. I have an open ticket regarding this issue but haven't heard anything back yet.

JDMurray · December 2018

I just installed Tor Browser 8.0.4 and "Error 1001" is still there for TE.

paul78 · December 2018

@Meggo - any news? Thanks.

Meggo · December 2018

@paul78 I just messaged them again about this on Monday. Here's what I got:

Looks like the issue is associated with a certain browser with a specific set of rules. I'll forward this over to my operations team to see if they can tell me more.

I just asked for an update and will let you know what I find out. I've been sharing this thread with them, so they have received all the background information.

paul78 · December 2018

Thanks @Meggo - I doubt that it's browser related - it seems to be DNS related through Tor network. The browser is based on Mozilla Firefox 60.4.0esr (64-bit).

One tidbit. I was just playing around and noticed that I don't have the same issue when accessing any other vanillacommunities.com hosted forums using Tor Browser. The only difference is that with TE, the subdomain community.infosecinstitute.com is a CNAME to the infosec.vanillacommunities.com. Perhaps the issue is with pairnic.com - I see that infosecinstitute.com's DNS is hosted there.

Meggo · December 2018

@paul78 Sharing this with some folks here now. Thanks for the tip!

paul78 · March 2019

@Meggo - any chance someone looked at the DNS integration issues with Cloudflare's TOR support? The errors are quite annoying. If you guys don't want to support access through TOR that's ok - I'm just wondering since Cloudflare has great TOR support.

Meggo · March 2019

@paul78 I'm sorry, I should have checked back in with you. I did not have any luck escalating this thru Vanilla. They attributed the issue to TOR specifically and recommended you use a more common browser like...Edge.

I don't recommend you use Edge, but the verdict is, the best way to avoid these issues for the time being is to not use TOR.

I'm sorry I don't have a better resolution for you, and appreciate all the research you did on this issue.

paul78 · March 2019

Thanks @Meggo.

BTW - the problem isn't with Vanilla. I can access other Vanilla sites hosted by vanillacommunities.com through the TOR network without any issues. I actually think the issue is related to Infosecinstitute's use of Pairnic's DNS services. It's not a browser issue - it's a network issue.

I have access to other different networks which seem to work. TOR network is just more convenient in general.

Thanks for checking back in. Cheers.

Meggo · March 2019

@paul78 I'll forward this to someone here who is much more informed than I and see if they can get a fix in place.

Meggo · March 2019

@paul78 I have an update and temporary fix (I hope). Here's what our IT department said:

We were able to recreate the issue with TOR, and also experienced this on a few other Vanilla-hosted sites like https://community.zteusa.com and https://community.phones.nokia.com. So far, the only temporary solution that seems to work is either by setting the ExitNodes to only use a specific country like "U.S." or try switching to "new circuit for this site". We're trying to identify if the issue has something to do with the exit node's DNS or a CloudFlare/Vanilla setting.

paul78 · March 2019

Thanks @Meggo - yeah - I usually have to create a new circuit whenever it fails. However, because of the nature of TOR, that means having to do that constantly.

The common thing about those other 2 sites that also have the same problem is the use of CNAME records. I noticed that forums which are directly accessible on vanillacommunities.com don't exhibit the same issue.

Thanks a bunch for following up.

paul78 · April 2019

BTW - I'm not sure if anything changed. But I now also get Cloudflare 1016 errors - https://support.cloudflare.com/hc/en-us/articles/234979888

Meggo · April 2019

@Infosec_Sam, any ideas?

Infosec_Sam · April 2019

Hmm, well here's what the error 1016 usually means:

Your Cloudflare DNS configuration does not have an A DNS record that matches the origin IP address.
You have a CNAME DNS record pointing to an external domain that cannot be resolved.
You're using Cloudflare Load Balancer and the origin host names (CNAMEs) in your default/region and fallback pools cannot be resolved via DNS. Use a fallback pool configured with an origin IP as a backup in case all other pools are unavailable.

The first two look like they would cause permanent issues only if we change something, but the third point might be a little more intermittent. If we're using Load Balancing, it might just be a little slow to switch from one pool to another, or the fallback pool isn't configured properly so every time it gets traffic, it throws it out.

JDMurray · April 2019

I would guess that the first two cases would produce obvious errors in CF's server logs that could be used to diagnose this problem. The third case looks like it would produce provider-side diagnostic messages too, but it's not clear if the events would be logged at the load balancers or the servers. This is important if the load balancers do not pass all network information along to the servers and therefore important diagnostic information is not logged.

In any case, if CF has logging turned on and all logged events are being correlated in a SIEM, they should be able to locate the errors based on @paul78 's ToR IP at the time the error occurred. If CF is interested in fixing this ToR connectivity problem, and @paul78 can recreate this error reliably, we should be able to open a ticket with CF and get @paul78 working with their tech support.

paul78 · April 2019

Thanks @JDMurray and @Infosec_Sam for the comments. I am kinda wondering if it's related to how vanillacommunities configure DNS - the multiple CNAME nesting could be introducing sufficient latency that the CF proxy is just timing out.

CF supports Tor very well and has some decent capabilities for Tor. The way that CF treats exit nodes is to consider them as individual separate countries. And CF updates all it's exit nodes every 15 minutes. So I don't think it's Tor exit nodes being blocked - if the exit nodes were blocked - I should either get a CAPTHA challenge or a CF 1009 error. And I've never seen either error.

CF does support Onion routing so that websites can be served directly on the Tor network. If TE wishes to support Tor, perhaps TE ought to consider exploring enabling this feature. Plus it generally provides better support to distinguish between humans and bots according to CF. And hey - don't all the cool infosec sites support access via Tor

Cloudflare 1001 DNS Errors

Comments