To Do

The next release of WhatWeb will be version 0.5.0. No release date is planned at this stage.

The WhatWeb 0.5.0-dev branch is available for the impatient.

---

WhatWeb Core

---

Rewrite Plugin Writing Guide

Include basic examples. Slowly introduce features. Provide tests to check that it works, ala rubykoans.

MD5 Matches

Implement aggressive md5 hash matches for version detection in popular open source software. WAFP and BlindElephant style.

Follow frames

Many websites still use frames on intro pages. A --follow-frames option would allow WhatWeb to grab these URLs instead of being stuck trying to fingerprint a HTML frameset.

--follow-frames=WHEN    Control when to follow frames. WHEN may be `never',
                        `frame-only', `iframe-only', `same-site', `same-domain'
                        or `always'. Default: never

Should frames be followed by default? Should following off-site frames be ignored or be a configurable option? Would never or same-site be the best default?

Aggression Level Five

Aggression level 5 will be used for requests which are directly related to compromising the security of a device or service without exploitation.

This includes requests designed to discover weak access controls caused by misconfiguration and requests designed to retrieve information from known information disclosure issues.

• Update Cisco-IOS plugin to determine if the device requires authorization using /level/[INTEGER]/exec/- requests
• Attempt login with default www-authorization credentials basic|digest base64(user:pass)
• Allow plugins to set custom headers, such as www-authenticate and cookies

Categorization

See the Categorization section on the Discussion page for more information, including suggested category names.

Goals of categorization:

• Run all plugins for a certain category:

./whatweb -c SCADA

• Display and group by categories in logging/output:

http://example.com [200]
HTML-Elements: Title, Meta-Generator, Script, Frame
Server: Apache
Web-App: Wordpress

• Multiple categories for plugins:

  • ordered by priority, ie, where conflicts exist in grouping, go with the first option.

category ["SCADA", "Device"]

• SSL Certificates

  • Extract hostname from ssl certificates
  • Should we test ciphers? Hmm.. probably not.

Plugin Errors and Crashes

These problems have not yet been allocated to a developer to be resolved.

Wordpress

bcoles: The wordpress plugin currently iterates through links and checks whether they're relative by checking for "/wp-content" or @base_uri + "/wp-content"

A cleaner solution would be using two regexps; one to compare @body to "/wp-content" and one to compare @body to @base_uri + "/wp-content"

This avoids having to convert long strings or potentially incompatible strings into URI objects.

$ ./whatweb 186.202.17.207 --debug
Exception `URI::InvalidURIError' at /usr/lib/ruby/1.8/uri/common.rb:436 - bad URI(is not URI?): http://www.tedxmataatlantica.com.br/wp-content/uploads/2011/05/Mata-Atlântica-195x110.jpg
Exception `URI::InvalidURIError' at /usr/lib/ruby/1.8/uri/generic.rb:732 - bad URI(is not URI?): http://www.tedxmataatlantica.com.br/wp-content/uploads/2011/05/Mata-Atlântica-195x110.jpg
ERROR: Plugin WordPress failed for http://186.202.17.207. bad URI(is not URI?): http://www.tedxmataatlantica.com.br/wp-content/uploads/2011/05/Mata-Atlântica-195x110.jpg
Exception `URI::InvalidURIError' at ./whatweb:751 - bad URI(is not URI?): http://www.tedxmataatlantica.com.br/wp-content/uploads/2011/05/Mata-Atlântica-195x110.jpg
/usr/lib/ruby/1.8/uri/generic.rb:732:in `merge': bad URI(is not URI?): http://www.tedxmataatlantica.com.br/wp-content/uploads/2011/05/Mata-Atlântica-195x110.jpg (URI::InvalidURIError)
        from /usr/lib/ruby/1.8/uri/common.rb:522:in `join'
        from /usr/lib/ruby/1.8/uri/common.rb:521:in `each'
        from /usr/lib/ruby/1.8/uri/common.rb:521:in `join'
        from ./plugins/wordpress.rb:63:in `passive'
        from ./plugins/wordpress.rb:61:in `each'
        from ./plugins/wordpress.rb:61:in `passive'
        from ./lib/plugins.rb:134:in `x'
        from ./whatweb:742:in `run_plugins'
        from ./whatweb:717:in `each'
        from ./whatweb:717:in `run_plugins'
        from ./whatweb:1374
        from ./whatweb:1253:in `initialize'
        from ./whatweb:1253:in `new'
        from ./whatweb:1253

HTML elements in the log file

HTML elements end up in the log file for a few reasons, as a result of:

• broken HTTP servers
• PHP errors/warnings in HTML format in the php warning or php error HTTP header(s)
• HTML in the x-powered-by HTTP header
• no HTTP headers at all
• maybe some SOAP/RPC services

You can find a few examples on ShodanHQ by searching for:

• href rackstar
• href netapp
• href -rackstar -netapp
• -HTTP -SIP -FTP -400 html

These searches sometimes yield interesting results, for example :

$ ./whatweb -p plugins-disabled/http-headers.rb http://194.245.135.17/
http://194.245.135.17/ [200] HTTP-Headers[connection: close,content-type: text/html,date: Tue, 15 Feb 2011 12:32:58 GMT,php warning: include(file:///C|/Dokumente und Einstellungen/Sabine/Anwendungsdaten/Macromedia/Dreamweaver 8/Configuration/ServerConnections/site/werbemittel/includes/application_bottom.php) %5B<a href='function.include'>function.include</a>%5D: failed to open stream: No such file or directory in /home/www/sites/194.245.135.17/site/werbemittel/index.php on line 64, include() %5B<a href='function.include'>function.include</a>%5D: Failed opening 'file:///C|/Dokumente und Einstellungen/Sabine/Anwendungsdaten/Macromedia/Dreamweaver 8/Configuration/ServerConnections/site/werbemittel/includes/application_bottom.php' for inclusion (include_path='.:/usr/local/php5/lib/php') in /home/www/sites/194.245.135.17/site/werbemittel/index.php on line 64,server: Apache/1.3.27 (Linux/SuSE) mod_throttle/3.1.2 mod_ssl/2.8.12 OpenSSL/0.9.6i mod_python/2.7.10 Python/2.3+ PHP/4.3.3 DAV/1.0.3,set-cookie: cookie_test=please_accept_for_session; expires=Thu, 17-Mar-2011 12:32:58 GMT; path=/; domain=abovogmbh.de,transfer-encoding: chunked,x-powered-by: PHP/5.2.6]