Quick and dirty script to check if a host is online via bash
SaltyHashes
Member Posts: 33 ■■■□□□□□□□
in Off-Topic
Here is a quick and dirty bash script that I authored to check to see if a host is online. This was tested using Ubuntu.
for word in $(cat domains.txt); do host $word; done > output.txt
This is assuming you have changed directories to the directory where the domains.txt file is located.
The point of this script is if you have large quantity of domains that you need to verify if they are online (active threat) and don't have the want or time to check each of them manually.
Please be advised that if you're editing in a Windows environment, the default line breaks are \r\n (carriage return and new line).
Linux environments only use \n (new line) and having \r (carriage returns) in your document will cause for this script to break.
Feel free to remove these carriage returns in an app like Notepad++.
Example of domains:
You can press Ctrl + H (or Search... > Replace). That will open this menu:
Replace all "\r" with "" (null) [ensure that Extended is selected]
The command should work fine in your bash terminal.
Example of output (viewing in vi):
for word in $(cat domains.txt); do host $word; done > output.txt
This is assuming you have changed directories to the directory where the domains.txt file is located.
The point of this script is if you have large quantity of domains that you need to verify if they are online (active threat) and don't have the want or time to check each of them manually.
Please be advised that if you're editing in a Windows environment, the default line breaks are \r\n (carriage return and new line).
Linux environments only use \n (new line) and having \r (carriage returns) in your document will cause for this script to break.
Feel free to remove these carriage returns in an app like Notepad++.
Example of domains:
You can press Ctrl + H (or Search... > Replace). That will open this menu:
Replace all "\r" with "" (null) [ensure that Extended is selected]
The command should work fine in your bash terminal.
Example of output (viewing in vi):
Comments
-
JDMurray
Admin Posts: 13,099 Admin
It appears the host command is only querying DNS servers. How is your script then determining if the specified hosts are active? -
SaltyHashes
Member Posts: 33 ■■■□□□□□□□
If the domain is suspended by the registrar, or the domain is not valid, you'll see a NXDOMAIN. (which I know you know)
This can save an analyst time by seeing which domains are live. -
JDMurray
Admin Posts: 13,099 Admin
You said the script was to check if a "host was online," not if a domain was still registered.
-
SaltyHashes
Member Posts: 33 ■■■□□□□□□□
I'm not understanding why you're trying to discredit my script. A server that has a domain registered but is offline will show a status of SERVFAIL.
This may not be a useful script for you, but this might be useful to someone.
The name of this post is not misleading... -
networker050184
Mod Posts: 11,962 Mod
That's not correct. The host command will still return if the device is offline. This does nothing to check if a host is actually online. A ping or nmap woud be a better idea to accomplish that.An expert is a man who has made all the mistakes which can be made. -
JDMurray
Admin Posts: 13,099 Admin
Yes, I'm not understanding how checking if a domain is registered using the 'host' command will also check if that domain is presently in use by any live hosts. I think we have different definitions of the condition "host is online." My definition is is akin with what networker said about ping and nmap. Maybe your definition might be related more to the passive collection of DNS traffic.
-
EANx
Member Posts: 1,077 ■■■■■■■■□□
I'll admit that my thoughts match the previous respondents. It's not that the script doesn't do something good, it's that other people don't agree with your definition of "host". I think you'll find that most people think of a host as a single device. That single device always has some method of addressing but it doesn't necessarily have a DNS record. My first thought was "what's wrong with ping?"
I think you'll also find that not getting emotionally involved with your work leads to better work. The work can be criticized without the author being criticized. -
cyberguypr
Mod Posts: 6,928 Mod
My quick and dirty script to check if a host is online is called PING.
Never mind me. I'll be in the corner over there with my popcorn. -
clarson
Member Posts: 903 ■■■■□□□□□□
not so sure a ping will tell you that a system is online.
take the wake on lan feature. the nic has power to receive the "magic packet" but the system can be off.
and I know that some nics implement a response to a ping on the interface card. nothing goes on a data bus or into memory or executed by the cpu.
the ping can be responded to by the nic even if the system is locked up, sitting at a blue screen of death, or powered down.
while a ping can tell you that there is a computer out there. it won't tell you that the OS is functioning.
maybe things aren't that way anymore. correct me if i'm wrong. -
JDMurray
Admin Posts: 13,099 Admin
A magic packet is a Layer 2/MAC frame that can only be sent over a LAN segment. Layer 2/MAC is implemented in the NIC's firmware and no OS is needed. Pinging for live hosts over the Internet requires a target running an OS with a TCP/IP stack to respond. Therefore, I would not expect a WakeOnLAN NIC to respond to a Layer 3/4 ICMP ECHO request.
There must be solutions to scan for WakeOnLAN adapters over the Internet, like we scan for SSH and VNC servers. -
clarson
Member Posts: 903 ■■■■□□□□□□
all nics implement layer 1 & 2. But some nics have functionality beyond layer 2. and expecting no nics to have that functionality could corrupt your data. and you won't know till a user calls in about their non functioning host. and you have a second problem to fix (the ping test is giving a false positive).
-
SaltyHashes
Member Posts: 33 ■■■□□□□□□□
Ping is not the answer for what an analyst is trying to accomplish here. Firewalls and routers can drop or not respond to ICMP packets.
The intent behind the creation of this script was to look at fraud domains that have malware, phishing, or tech scams on them at scale. Web browsers are reliant on DNS to resolve the host for which the web content is on. That's why the host command was appropriate in this scenario.
If an analyst is tasked with looking at 200 domains to find additional evidence, this script would report back which domains were accessible.
If this script can show that perhaps 35 of the domains in that list are not active, that can save considerable amount of time. -
paul78
Member Posts: 3,016 ■■■■■■■■■■
Personally, I prefer to just use a script that would telnet to iana.org's whois server to check for registered domains or just use whois. But using DNS to check for a registered domain works too, I suppose. @OP - most people suggested ping because you are saying that this checks to see if a host is online. Theorically - you could still do it with ping because the DNS query would fail with ping just as host would. Although if you are simply doing a domain check, using DNS is simpler.
-
SaltyHashes
Member Posts: 33 ■■■□□□□□□□
Personally, I prefer to just use a script that would telnet to iana.org's whois server to check for registered domains or just use whois. But using DNS to check for a registered domain works too, I suppose. @OP - most people suggested ping because you are saying that this checks to see if a host is online. Theorically - you could still do it with ping because the DNS query would fail with ping just as host would. Although if you are simply doing a domain check, using DNS is simpler.
Paul, thank you for the feedback. Absolutely this script could be modified to perform other actions such as telnet, whois, dig, ping, etc.
Do you/your team use a script that performs a telnet request to IANA? Also, what are the advantages of doing that?
Our team and our network are not fond of opening telnet connections. Could ssh be used instead? -
paul78
Member Posts: 3,016 ■■■■■■■■■■
Nope - it's not the type of work that we do now. But I used to have research team that would do related stuff - but for other reasons. We were doing stuff at scale so we tended to code in golang but I have a preference for rust.SaltyHashes wrote: »Do you/your team use a script that performs a telnet request to IANA? Also, what are the advantages of doing that?
I mentioned telnet only because it can be used to make a plain tcp connect. You could just use netcat instead to query a whois server. You cannot use ssh because an ssh client that will start the ssh negotiation.SaltyHashes wrote: »Our team and our network are not fond of opening telnet connections. Could ssh be used instead?
Whois is a minimalist protocol. It's just a tcp listener on port 43 that takes a newline terminated string and spits out it's records. So you can do something as simple as:
echo 'techexams.net' | netcat whois.iana.org 43
I'm somewhat curious as to why you are doing domain registration checks. Like @JDMurray - I originally thought you were doing some sort of targeted passive DNS collection. Passive DNS collection can be challenging to do at scale. And there are only a few players out there that have datasets of significance - plus it's tough to monetize those data set. -
JDMurray
Admin Posts: 13,099 Admin
I've been using the whosip command line tool from Nirsoft for years. It queries WHOIS servers via port 43/tcp. Nicely formatted output too.
-
SaltyHashes
Member Posts: 33 ■■■□□□□□□□
I've been using the whosip command line tool from Nirsoft for years. It queries WHOIS servers via port 43/tcp. Nicely formatted output too.
@JDMurray, Thank you for the pointer. Going to have to add this to my toolkit. Huge fan of freeware too. The CIDR field is a nice addition to standard whois output.
If you have any suggestions to improve the one-line script, I'm all ears. Glad we have hashed-out our differences on how we view the word "host".
