Python IoT search with Wget and Yara Rules

# Given a list of urls, determine what type of IoT device (or any device for that matter) they are based on you plugging in Yara rules into the .yar files

import os
import subprocess
import traceback

debug = "false"
skipDownload = "false"
input = "urls.txt"
yaraSpecificRuleFile = "IoTSpecific.yar"
yaraGenericRuleFile = "IoTGeneric.yar"
wgetParams = "--quiet --no-check-certificate --timeout=2 --tries=3"
yaraParams = ""
outputExt = ".html"
urls = open(input, "r")

if skipDownload == "false":
 for url in urls:
  url = url.rstrip()
  cleanurl = url.rstrip().replace('/','_').replace('\\','_').replace(':','_').replace('.','_').replace('&','_').replace('?','_').replace('=','_').replace('%','_') + outputExt
  wgetCommand = ("wget %s --output-document=%s %s 2>/dev/null" % (wgetParams, cleanurl, url))
  if(debug == "true"):
   print(("DEBUG,Starting Download of '%s' to '%s'" % (url, cleanurl)))
  try:
   output = subprocess.check_output(wgetCommand, shell=True)
   if "error" in output:
    print(("ERROR,Unable to download '%s' error '%s'" % (wgetCommand, output)))
  except Exception:
   print(("ERROR,Unable to download '%s' error '%s'" % (wgetCommand,traceback.print_exc())))
  if(debug == "true"):
   print(("DEBUG,Finished Download of '%s' to '%s'" % (url, cleanurl)))

for htmlfile in os.listdir('.'):
 if htmlfile.endswith(".html"):
  if os.stat(htmlfile).st_size == 0:
   print(("NoResponse,%s" % (htmlfile)))
  else:
   yaraCommand = ("yara %s %s %s" % (yaraParams, yaraSpecificRuleFile, htmlfile))
   if(debug == "true"):
    print(("DEBUG,Starting Scanning: '%s'" % (htmlfile)))
   try:
    output = subprocess.check_output(yaraCommand, shell=True)
    if "error" in output:
     print(("ERROR,Unable to Scan '%s' error '%s'" % (yaraCommand, output)))
    if output:
     output = output.rstrip()
     print(output.replace(" ",","))
    else:
     yaraRescanCommand = ("yara %s %s %s" % (yaraParams, yaraGenericRuleFile, htmlfile))
     if(debug == "true"):
      print(("DEBUG,Starting ReScanning: '%s'" % (htmlfile)))
     try:
      output = subprocess.check_output(yaraRescanCommand, shell=True)
      if "error" in output:
       print(("ERROR,Unable to ReScan '%s' error '%s'" % (yaraCommand, output)))
      if output:
       output = output.rstrip()
       print(output.replace(" ",","))
      else:
       print(("NoMatch,%s" % (htmlfile)))
     except Exception:
      print(("ERROR,Unable to ReScan '%s' error '%s'" % (yaraCommand,traceback.print_exc())))
     if(debug == "true"):
      print(("DEBUG,Finished ReScanning: '%s'" % (htmlfile)))
   except Exception:
    print(("ERROR,Unable to Scan '%s' error '%s'" % (yaraCommand,traceback.print_exc())))
   if(debug == "true"):
    print(("DEBUG,Finished Scanning: '%s'" % (htmlfile)))

urls.close()

wget -i Urls.txt

download a list of #phishingkit zips put files into urls.txt run command wget -i urls.txt

Snort Rules Monitoring User-Agents

I think some Snort rules like these could be used to monitor specific user-agents that sometimes are common with recon, vulnerability scans, and exploits.

WPScan

alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS (sid:xxx; gid:1; flow:established,to_server; content:"WPScan"; http_header; fast_pattern:only; pcre:"/^User\x2dAgent\x3a\x20[^\r\n]*WPScan/Hm"; metadata:service http; msg:"BLACKLIST User-Agent known malicious user-agent string WPScan - vulnerability scanner"; classtype:network-scan; rev:1; )

Wget

alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS (sid:xxx; gid:1; flow:established,to_server; content:"Wget"; http_header; fast_pattern:only; pcre:"/^User\x2dAgent\x3a\x20Wget/Hm"; metadata:service http; msg:"BLACKLIST User-Agent known malicious user-agent string Wget non-browser"; classtype:network-scan; rev:1; )

Synapse

alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS (sid:xxx; gid:1; flow:established,to_server; content:"Synapse"; http_header; fast_pattern:only; pcre:"/^User\x2dAgent\x3a\x20[^\r\n]*Synapse/Hm"; metadata:service http; reference:url,http://www.spambotsecurity.com/forum/viewtopic.php?f=43&t=2876; msg:"BLACKLIST User-Agent known malicious user-agent string Synapse - SQLi IoC"; classtype:network-scan; rev:1; )

SqlMap

alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS (sid:xxx; gid:1; flow:established,to_server; content:"sqlmap"; http_header; fast_pattern:only; pcre:"/^User\x2dAgent\x3a\x20[^\r\n]*sqlmap/Hm"; metadata:service http; reference:url,http://sqlmap.org/; msg:"BLACKLIST User-Agent known malicious user-agent string sqlmap - vulnerability scanner"; classtype:network-scan; rev:1; )

Python

alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS (sid:xxx; gid:1; flow:established,to_server; content:"Python-urllib"; http_header; fast_pattern:only; pcre:"/^User\x2dAgent\x3a\x20Python/Hm"; metadata:service http; msg:"BLACKLIST User-Agent known malicious user-agent string Python non-browser"; classtype:network-scan; rev:3; )

PycURL

alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS (sid:xxx; gid:1; flow:established,to_server; content:"PycURL"; http_header; fast_pattern:only; pcre:"/^User\x2dAgent\x3a\x20[^\r\n]*PycURL/Hm"; metadata:service http; reference:url,http://pycurl.io/; msg:"BLACKLIST User-Agent known malicious user-agent string PycURL - non Browser"; classtype:network-scan; rev:1; )

Paros

alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS (sid:xxx; gid:1; flow:established,to_server; content:"Paros"; http_header; fast_pattern:only; pcre:"/^User\x2dAgent\x3a\x20[^\r\n]*Paros/Hm"; metadata:service http; reference:url,http://sectools.org/tool/paros/; msg:"BLACKLIST User-Agent known malicious user-agent string Paros - vulnerability scanner"; classtype:network-scan; rev:1; )

OpenVAS

alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS (sid:xxx; gid:1; flow:established,to_server; content:"OpenVAS"; http_header; fast_pattern:only; pcre:"/^User\x2dAgent\x3a\x20[^\r\n]*OpenVAS/Hm"; metadata:service http; reference:url,http://www.openvas.org/; msg:"BLACKLIST User-Agent known malicious user-agent string OpenVAS - vulnerability scanner"; classtype:network-scan; rev:2; )

Nmap

alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS (sid:xxx; gid:1; flow:established,to_server; content:"Nmap"; http_header; fast_pattern:only; pcre:"/^User\x2dAgent\x3a\x20[^\r\n]*Nmap/Hm"; metadata:service http; reference:url,https://nmap.org/book/nse.html; msg:"BLACKLIST User-Agent known malicious user-agent string Nmap - scanner"; classtype:network-scan; rev:2; )

Nikto

alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS (sid:xxx; gid:1; flow:established,to_server; content:"Nikto"; http_header; fast_pattern:only; pcre:"/^User\x2dAgent\x3a\x20[^\r\n]*Nikto/Hm"; metadata:service http; reference:url,http://sectools.org/tool/nikto/; msg:"BLACKLIST User-Agent known malicious user-agent string Nikto - vulnerability scanner"; classtype:network-scan; rev:1; )

Kazehakase

alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS (sid:xxx; gid:1; flow:established,to_server; content:"Kazehakase"; http_header; fast_pattern:only; pcre:"/^User\x2dAgent\x3a\x20[^\r\n]*Kazehakase/Hm"; metadata:service http; reference:url,https://en.wikipedia.org/wiki/Kazehakase; msg:"BLACKLIST User-Agent known malicious user-agent string Kazehakase - suspicious browser"; classtype:network-scan; rev:1; )

curl

alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS (sid:xxx; gid:1; flow:established,to_server; content:"curl"; http_header; fast_pattern:only; pcre:"/^User\x2dAgent\x3a\x20[^\r\n]*curl/Hm"; metadata:service http; msg:"BLACKLIST User-Agent known malicious user-agent string curl - non browswer"; classtype:network-scan; rev:1; )

More about neonprimetime

• neonprimetime pastebin
• neonprimetime virustotal
• @neonprimetime twitter
• neonprimetime reddit

Top Blogs of all-time

1. pagerank botnet sql injection walk-thru
2. DOM XSS 101 Walk-Through
3. An Invoice email and a Hot mess of Java

Top Github Contributions

1. Qualys Scantronitor 2.0

Copyright © 2016, this post cannot be reproduced or retransmitted in any form without reference to the original post.

stssys.htm wget request

GET /stssys.htm HTTP/1.0
User-Agent: Wget(linux)


Saw this request in the http logs, what could it be? It appears in 2013 TRENDnet Print Server vulnerability that allowed an attacker to reset the printer to factory defaults which would then allow them to set the new IP address. This could be done remotely without authentication by acccessing particular web pages. This attacker is just using the linux wget command to scan the internet and look for vulnerable print servers.

More about neonprimetime

• neonprimetime pastebin
• neonprimetime virustotal
• @neonprimetime twitter
• neonprimetime reddit

Top Blogs of all-time

1. pagerank botnet sql injection walk-thru
2. DOM XSS 101 Walk-Through
3. An Invoice email and a Hot mess of Java

Top Github Contributions

1. Qualys Scantronitor 2.0

Copyright © 2016, this post cannot be reproduced or retransmitted in any form without reference to the original post.