v4.2.0

Primary fix is for the --max-targets flag when used with percentages of the address space and multiple ports. Includes a few other improvements as well, see below:

What's Changed

• Add dependency checks to CLists for better UX by @phillip-stephens in #883
• Update README.md by @zakird in #884
• the --max-targets flag with a percent wasn't respecting multi-ports by @phillip-stephens in #886
• Updated changelog/readme to bump version to 4.2 by @phillip-stephens in #887
• Bump docker base image and relevant packages by @phillip-stephens in #888

Full Changelog: v4.1.1...v4.2.0

v4.1.1

This patch release updates the documentation in CHANGELOG.md and bumps the latest version in README.md

v4.1.0

ZMap 4.1.0 introduces the notion of multi-port scanning, which has been a long requested feature. This is a breaking change since ZMap now operates on a metric of (ip,port) target instead of simply IP (e.g., for scan rate). It also introduces new dependencies (e.g., libjudy) to support multi-port scanning and changes ZMap's command-line interface.

Other major changes include:

• Randomize the IP packet ID to prevent fingerprinting of scan traffic
• Add support for Netmap to increase performance on supported NIC's w/ the requisite drivers
• Add hardware acceleration for AES to improve performance when the CPU begins to become the bottleneck
• Added integration tests and compilation checks for supported OS's as Github Actions
• Added --probe-args options to the TCP SYN scan module to send TCP header options identical to Ubuntu, MacOS, Windows (default), or minimum sized frames (with MSS only, fit within a minimum sized Ethernet payload). Windows offers a slightly higher hitrate with modest increase in bandwidth requirements.
• Various other bug fixes and enhancements

Thanks to our community for the many improvements and fixes that made ZMap v4.1.0 possible!

Full Changeless

• Store link-layer timestamp in icmp_echo_time module by @maxmouchet in #726
• Port independence by @zakird in #727
• Multi-Port Scanning by @zakird in #728
• Update README.md by @zakird in #729
• Bugfix: Passing no port to ICMP module (or any module without a port requirement) causes segfault by @zakird in #731
• Warning about packet streams and fixed dst port validation by @zakird in #738
• Fixed typos in man pages and --help text by @phillip-stephens in #739
• Update manual pages to add details to --probes to contrast with --retries by @phillip-stephens in #740
• updated Fedora dependencies to having Judy-devel since the prior pkg doesn't exist in latest fedora by @phillip-stephens in #741
• phillip/fix-bug-cmake-lists: fixed hard-coded path by @phillip-stephens in #742
• Link FAQ and Discussions resources in Issue Template and README by @phillip-stephens in #745
• Use interface with default route on Linux by @Tim--- in #733
• Update --help text to use new --allowlist-file parameter by @phillip-stephens in #750
• Add GitHub Wiki Getting Started Guide to README.md by @phillip-stephens in #749
• Set default senders as min(4, number of cores on host) by @phillip-stephens in #752
• Align --retries default with --help and manual pages by @phillip-stephens in #755
• Add dynamic packet batching to take advantage of sendmmsg on Linux by @phillip-stephens in #751
• Small README Updates by @zakird in #760
• Gracefully handle IPv6 addresses in blocklist.conf by @phillip-stephens in #759
• Don't error in send_cleanup with empty batches (like with --dryrun) by @phillip-stephens in #766
• Add option to change "recursion desired" bit in DNS probe module by @gutjuri in #764
• Prevent --batch from overflowing the uint8_t it is being stored in by @droe in #770
• Multiple fixes for BSD get_gateway code, remove libdnet by @droe in #772
• Split send-bsd.h into send-mac.h and send-bsd.h and add GitHub action for compiling on FreeBSD/MacOS by @phillip-stephens in #771
• Fix input handling around -P flag with the DNS module, Resolves #746 by @phillip-stephens in #757
• Stop defining the now defunct _SYSTYPE_BSD by @droe in #774
• Fix mem leak on BSD get_hw_addr failure path by @droe in #777
• Support IP layer mode on macOS by @droe in #776
• Fix size calculations to prevent overflowing fake_eth_hdr in IP layer mode by @droe in #775
• phillip/748: resolves #748 where using the -I CLI flag would cause an assertion failure whenever it was run by @phillip-stephens in #753
• Fix issue in BSD get_hw_addr where last ARP entry would be skipped by @droe in #779
• Add GitHub CI checks for Debian, Gentoo, Arch, CentOS, and Fedora by @phillip-stephens in #780
• Fix benign compiler warnings on macOS/FreeBSD clang by @droe in #782
• Avoid isomorphism 0 by @droe in #784
• phillip/781: elevate debug log if can't get a socket on bsd to fatal by @phillip-stephens in #783
• Support IP layer mode on FreeBSD and merge Mac with BSD send code by @droe in #778
• Remove the Gentoo OS Github Compilation Check by @phillip-stephens in #785
• Fix assertion failure when shard_roll_to_valid() iterates to the end by @droe in #786
• Add support for very fast packet I/O using netmap(4) on FreeBSD by @droe in #787
• Use stdbool instead of typedef uint8_t bool in dns probe module by @droe in #790
• Fix two crashes in dns probe module args handling by @droe in #791
• Fixed bug in the shard initialization w/ multi-port by @phillip-stephens in #792
• Move send code from headers to their own compilation units by @droe in #793
• Add a suite of integration tests to Github Actions by @phillip-stephens in #789
• Removed unused idx variable by @phillip-stephens in #794
• Bump batch size from uint8_t to uint16_t by @droe in #797
• netmap: Add --netmap-wait-ping to work around STP blocking ports temporarily by @droe in #796
• Added info about users with a VPN to error by @phillip-stephens in #795
• Port netmap code to Linux by @droe in #800
• Add --probe-args for TCP SYN module to send packets with the various sets of TCP options by @phillip-stephens in #799
• Double aesrand_getword() performance by using full 128 bits by @droe in #801
• Update README.md by @zakird in #802
• Fix a number of benign compiler warnings on Linux/GCC by @droe in #805
• Prevent fingerprinting ZMap by randomizing the IP ID by @phillip-stephens in #798
• Generalise zqueue to non-string data pointers, fix memory leaks by @droe in #806
• Probe-modules: IPIP by @yannayl in #684
• netmap: Reply to ARP requests from gateway for scan source IPs by @droe in #807
• Netmap: fixed ubuntu compilation error by added strlcpy definition to utility.c by @phillip-stephens in #808
• AES hardware acceleration using AES-NI and ARMv8 CE by @droe in #803
• Testing: added compilation tests for all Ubuntu LTS versions >= 16.04 by @phillip-stephens in #810
• Improve netmap docs by @droe in #813
• Added upper bound on number of threads ZMap can use by @phillip-stephens in #811
• Fix use-after-free's in IPIP probe module by @droe in #815
• Do not skip elements >= (1 << 48) w/o checking for sentinel by @droe in #814
• Bump -std to gnu11 for static assertions by @droe in #817
• Smaller MAX_PACKET_SIZE and better batch cache locality by @droe in #816
• Less copying, more batch optimisation by @droe in #820
• Fix being unable to open an IP layer socket on Linux by @phillip-stephens in #824
• Don't tell user to use 0 cores on machines with only 1 core by @phillip-stephens in #822
• Error out if user uses > 255 Sending Threads by @phillip-stephens in #826
• Fix more mostly benign Linux/GCC compiler warnings by @droe in #827
• Capped percent complete in monitor.c by @phillip-stephens in #825
• Output the progress once more when the whole scan process finished by @WangYihang in #828
• Added CHANGELOG for releases v. 4.0 and 4.1 by @phillip-stephens in #834
• Fix incorrect status ETA and percent complete for multi-port scans with --list-of-ips and --rate by @phillip-stephens in #835
• Clang Format by @zakird in #838
• Allow small scans without erroring by @zakird in #839
• Fix typo in --help text by @phillip-stephens in https://github....

v4.1.0-RC-4

This bug release version contains a fix for a breaking bug that would cause the receive thread to become stuck processing a TCP SYN scan packet if it came back with a malformed TCP option. This would happen randomly, but almost always within 100k-100M random hosts scanned.

v4.1.0-RC-3

This release features several bug fixes and optimizations since v4.1.0-RC-2

Enhancements

• Use same IP TTL as ubuntu (#850)
• Add TCP options parsing in receive thread (#858)

Bug Fixes

• fix inaccurate max_packet_length with TCP options (#853)
• Fix inaccurate hit-rate printed during scan when using output filtering (#864)

Optimizations

• use existing timestamp info in libpcap to avoid the system call (#848)
• Increase the slow_start definition to 200 pps, using less CPU at lower send rates (#851)
• Reduce memory used for radix cache (#860)

ZMap v4.1.0-RC2

This release contains several bug fixes found since tagging v4.1.0-RC1:

1. Inaccurate estimated time remaining and percentage complete calculations during a multi-port scan
2. Fixed building from source on MidnightBSD
3. Fixed hitrate calculation with multiple --probes packets per target

ZMap v4.1.0-RC1

This release has several performance improvements and many bug fixes for various reported issues. We'll push this to the various OS package managers in about a month barring any reported issues!

Changes

• Randomize the IP packet ID to prevent fingerprinting of scan traffic
• Add support for Netmap to increase performance on supported NIC's w/ the requisite drivers
• Add hardware acceleration for AES to improve performance when the CPU begins to become the bottleneck
• Added integration tests and compilation checks for supported OS's as Github Actions
• Added --probe-args options to the TCP SYN scan module to send TCP header options identical to Ubuntu (default), MacOS, Windows, or No Options.
• Various other bug fixes and enhancements

Thanks to everyone who helped contribute features towards this release!
cc: @droe @WangYihang @gutjuri @zakird

ZMap 4.0.0 RC1

ZMap 4.0.0 (RC1) introduces the notion of multi-port scanning, which has been a long requested feature. This is a breaking change since ZMap now operates on a metric of (ip,port) target instead of simply IP (e.g., for scan rate). It also introduces new dependencies (e.g., libjudy) to support multi-port scanning and changes ZMap's command-line interface.

Features:

• Multi-port scanning support
• Store link-layer timestamp in icmp_echo_time module (#726)
• Build support for ARM-based Macs

ZMap 3.0.0

We're happy to provide ZMap 3.0.0, only slightly under six years late. We recommend using this release over any previous 2.x release.

ZMap 3.0.0 represents several years of development and contains more than a hundred small bug fixes from ZMap 2.1.1., including many fixes for UDP modules, sharding, and progress calculation. Below, are some of the most important changes:

Bugs:

• Fix send rate calculations
• Accept RST packets for SEQ+0 (per RFC)
• Packets per second is packets per second now instead of IPs per second
• MaxResults is now the number of packets that pass the output filter (#502)
• Try all routing tables in Linux
• Fix crash on invalid UDP packets
• Fix failed initialize on single-question DNS probes
• Fix inaccurate blocklist warning
• Use monotonic OS clocks for monitoring and rate estimation
• Fix bugs in UDP template arguments
• Increase UDP PCAP snaplen to prevent packet truncation
• Exit on failed sends
• Fix incorrect time remaining calculations on sharded scans

Features:

• Added --list-of-ips feature which allows scanning a large number (e.g., hundreds of millions or billons) of individual IPS
• Improved user messages when network settings can't be automatically discovered
• Consistent ICMP support and handling across all probe modules (#470)
• Set TCP MSS flags to avoid filtering by destination hosts (#673)
• Sane default behavior that can be explained with other CLI flags
• Non-Flat Result output and JSON result encoding
• IP Fragment Checking
• DNS, TCP SYN-ACK, and Bacnet Probe Module
• Change Whitelist/Blacklist terms to Allowlist/Blocklist
• Add extended validation bytes for probe modules that can use greater entropy
• Support non-continuous source IP's (#516)
• Add NetBSD and DragonFly BSD compatibility code (#411)
• Improved ICMP validation based on returned packet (#419)

Removed Functionality:

• Drop Redis and MongoDB support (#661)

ZMap 3.0.0 RC2

Fix bug in monitor.c that incorrectly reported time remaining.