Dastardly Crashed Server

[Sh4d0wHunt3rX]

I was doing a very long scan on Dell

Started: 2024-07-24 06:52:15,380
Crashed: 2024-07-25 06:54:59,451

When Baddns finished, Dastardly started heavily.

So, for some seconds, my terminal was not responsive anymore, however, server was live. After few seconds, it became ok, I checked btop and saw this:

Then the ram became better in range of 70-90%. However, after about 15 minutes, suddenly I got disconnected from Termius and couldn't connect anymore.

For ram warning, I got only these 2 in debug:

2024-07-25 06:41:37,097 [WARNING] bbot.scanner scanner.py:1051 System memory is at 96.3% (595.96MB remaining)
2024-07-25 06:41:52,668 [WARNING] bbot.scanner scanner.py:1051 System memory is at 90.2% (1.53GB remaining)

Last lines

2024-07-25 06:54:59,450 [INFO] bbot.scanner scanner.py:1031 2024-07-24_06-52-07: Modules running (incoming:processing:outgoing) smuggler(7,984:1:0), dastardly(5,583:1:0), excavate(2,912:8:1,000), baddns(86:8:0), dnsresolve(10:20:0)

2024-07-25 06:54:59,450 [INFO] bbot.scanner scanner.py:1031 2024-07-24_06-52-07: Events produced so far: RAW_DNS_RECORD: 450930, DNS_NAME: 243527, IP_ADDRESS: 30027, VHOST: 20385, SOCIAL: 13258, URL_UNVERIFIED: 8867, OPEN_TCP_PORT: 5478, WEB_PARAMETER: 5249, FINDING: 4571, HTTP_RESPONSE: 4430, URL: 3137, TECHNOLOGY: 2297, WEBSCREENSHOT: 1189, WAF: 1057, FILESYSTEM: 829, VULNERABILITY: 508, ASN: 429, STORAGE_BUCKET: 187, CODE_REPOSITORY: 178, EMAIL_ADDRESS: 74, PROTOCOL: 64, ORG_STUB: 2, SCAN: 1, DNS_NAME_UNRESOLVED: 1

2024-07-25 06:54:59,451 [INFO] bbot.scanner scanner.py:1031 2024-07-24_06-52-07: 977,669 events in queue (3,648 processed in the past minute)

Server rebooting time:

Wed 2024-07-24 06:44:08 CEST—Thu 2024-07-25 06:55:14 CEST
Thu 2024-07-25 06:55:25 CEST

So, I'm not sure if I interrupted BBOT by restarting the server or it already crashed the server.

I need to know:

1. Is it possible Dastardly actually crash the server? How to prevent it?
2. Is it possible that I can't connect to server because of ram usage but the server be live and bbot is still running?

Thanks 🙏

[TheTechromancer]

Yes it's definitely possible. If the server crashed, it was almost definitely because it ran out of memory. From the screenshot, you can see that dastardly spawns chromium processes as part of its scan, and Chrome is infamously a memory hog. If you don't want to disable dastardly, you will need to give the server more RAM, and probably create a fat swapfile for safe measure:

# Create a file which will be used for swap.
# NOTE: if this doesn't work, use `dd if=/dev/zero of=/swapfile bs=1024 count=1048576`
fallocate -l 5G /swapfile

# Set the correct permissions.
chmod 600 /swapfile

# Set up a Linux swap area
mkswap /swapfile

# Activate swap
swapon /swapfile

# Add in `/etc/fstab`
echo '/swapfile swap swap defaults 0 0' >> /etc/fstab

# Verify the swap status.
swapon --show

[TheTechromancer]

From your screenshot it looks like BBOT is taking up close to 8GB by itself. That is pretty surprising. I'd be curious to know what version of BBOT and what modules you're using in that scan.

The biggest culprit for memory usage in BBOT was always massdns, because it needed to keep track of its mutations. But in BBOT 2.0, with dnsbrute_mutations, its footprint should be a lot smaller (max of 100MB) because it uses a bloom filter instead.

So if you are running BBOT 2.0, that memory usage is very strange, and might indicate a bug.

[Sh4d0wHunt3rX]

Thanks for the info about swap, I will give it a try : ) Is it possible to limit dastardly ram usage somehow? Or BBOT is already doing this.

My targets were:

dell.com
delltechnologies.com

Duration 24 hours with 15k hostnames.

BBOT v2.0.0.4272rc

2024-07-24 06:52:15,380 [TRACE] bbot.scanner scanner.py:1071 Ran BBOT v2.0.0.4272rc at 2024-07-24 06:52:15.379378, command: /root/.local/bin/bbot -t /root/.bbot/scans/Dell/project-settings/Dell-whitelists.txt -m affiliates anubisdb asn baddns baddns_zone badsecrets bevigil binaryedge builtwith c99 censys certspotter chaos columbus crobat crt dnscaa digitorus dnscommonsrv dnsdumpster filedownload fingerprintx fullhunt git github_codesearch gowitness hackertarget httpx internetdb ipneighbor leakix dnsbrute dnsbrute_mutations myssl portscan oauth otx passivetotal postman rapiddns riddler robots securitytrails secretsdb shodan_dns sitedossier sslcert subdomaincenter sublist3r threatminer urlscan viewdns virustotal wafw00f wappalyzer wayback zoomeye hunt paramminer_headers dastardly nuclei vhost wpscan ajaxpro host_header iis_shortnames ntlm smuggler telerik url_manipulation trufflehog -om asset_inventory subdomains -c omit_event_types=[DNS_NAME_UNRESOLVED,URL_UNVERIFIED] url_extension_httpx_only=[] url_querystring_remove=false modules.baddns.only_high_confidence=true -y --allow-deadly --blacklist /root/.bbot/scans/Dell/project-settings/Dell-blacklists.txt -o /root/.bbot/scans/Dell -n 2024-07-24_06-52-07

[TheTechromancer]

Thanks. It's not possible to limit dastardly's memory usage.

Do you have the debug status message that includes each module's memory usage?

[Sh4d0wHunt3rX]

Unfortunately, I have only normal bbot debug file. Can you tell me how I can get "debug status message that includes each module's memory usage" ? Maybe next time I can do that.

[TheTechromancer]

it's the normal status message that prints whenever you have --debug enabled. It prints every 15 seconds.