* fix(http): interactsh matching with `payloads`
in parallel execution.
Templates using `payloads` with Interactsh
matchers failed to detect OAST interactions
because the parallel HTTP execution path (used
when `payloads` are present) did not register
Interactsh request events, unlike the seq path.
This caused incoming interactions to lack
associated request context, preventing matchers
from running and resulting in missed detections.
Fix#5485 by wiring
`(*interactsh.Client).RequestEvent` registration
into the parallel worker goroutine, make sure both
execution paths handle Interactsh correlation
equally.
Signed-off-by: Dwi Siswanto <git@dw1.io>
* test: add interactsh with `payloads` integration
Signed-off-by: Dwi Siswanto <git@dw1.io>
* test: disable interactsh-with-payloads
Signed-off-by: Dwi Siswanto <git@dw1.io>
---------
Signed-off-by: Dwi Siswanto <git@dw1.io>
The `race` condition directive was broken due to
a strict dependency on `threads > 0` for parallel
execution, causing templates with `race` directive
enabled but no explicit threads to fall back to
seq execution.
This regression was introduced in v3.2.0 (#4868),
which restricted parallel execution to only when
`payloads` were present.
Fixes#5713 to allow race conditions even w/o
explicit `payloads`, and add a default thread
count when race is enabled but threads is 0.
Signed-off-by: Dwi Siswanto <git@dw1.io>
`MergeMaps` accounts for 11.41% of allocs (13.8
GB) in clusterbomb mode. With 1,305 combinations
per target, this function is called millions of
times in the hot path.
RCA:
* Request generator calls `MergeMaps` with single
arg on every payload combination, incurring
variadic overhead.
* Build request merges same maps multiple times
per request.
* `BuildPayloadFromOptions` recomputes static CLI
options on every call.
* Variables calls `MergeMaps` $$2×N$$ times per
variable evaluation (once in loop, once in
`evaluateVariableValue`)
Changes:
Core optimizations in maps.go:
* Pre-size merged map to avoid rehashing (30-40%
reduction)
* Add `CopyMap` for efficient single-map copy
without variadic overhead.
* Add `MergeMapsInto` for in-place mutation when
caller owns destination.
Hot path fixes:
* Replace `MergeMaps(r.currentPayloads)` with
`CopyMap(r.currentPayloads)` to eliminates
allocation on every combination iteration.
* Pre-allocate combined map once, extend in-place
during `ForEach` loop instead of creating new
map per variable (eliminates $$2×N$$ allocations
per request).
Caching with concurrency safety:
* Cache `BuildPayloadFromOptions` computation in
`sync.Map` keyed by `types.Options` ptr, but
return copy to prevent concurrent modification.
* Cost: shallow copy of ~10-20 entries vs. full
merge of vars + env (85-90% savings in typical
case)
* Clear cache in `closeInternal()` to prevent
memory leaks when SDK instances are created or
destroyed.
Estimated impact: 40-60% reduction in `MergeMaps`
allocations (5.5-8.3 GB savings from original
13.8 GB). Safe for concurrent execution and SDK
usage with multiple instances.
Signed-off-by: Dwi Siswanto <git@dw1.io>
This patch enables TLS session resumption by
setting a shared LRU session cache
(`ClientSessionCache`) in all HTTP client TLS
configs. This reduces handshake overhead and CPU
usage for repeated conns to the same host,
improving throughput and efficiency in
clusterbomb/pitchfork modes.
This applied to HTTP-request-based and headless-
request-based protocols.
No runtime/compatibility impact.
Signed-off-by: Dwi Siswanto <git@dw1.io>
The "Skipped X from target list as found
unresponsive permanently" message was logged on
every `(*Cache).Check()` call for hosts with
permanent errors, resulting in thousands of
duplicate log entries in verbose mode.
Wrap the log statement in `sync.Once` to match the
behavior already used for non-permanent error
logging.
Signed-off-by: Dwi Siswanto <git@dw1.io>
Continue the fix from #6666 by converting
remaining direct Body assignments to use setter
methods:
* pkg/fuzz/component/body.go:139: use
`SetBodyReader()` in transfer-encoding path.
* pkg/protocols/http/request.go:694: use
`SetBodyString()` in fuzz component `Rebuild()`.
Fixes#6692.
Signed-off-by: Dwi Siswanto <git@dw1.io>
* fix(http): pass `dynamicValues` to `EvaluateWithInteractsh`
When `LazyEval` is true (triggered by `variables`
containing `BaseURL`, `Hostname`,
`interactsh-url`, etc.), variable expressions are not
eval'ed during YAML parsing & remain as raw exprs
like "{{rand_base(5)}}".
At request build time, `EvaluateWithInteractsh()`
checks if a variable already has a value in the
passed map before re-evaluating its expression.
But, `dynamicValues` (which contains the template
context with previously eval'ed values) was not
being passed, causing exprs like `rand_*` to be
re-evaluated on each request, producing different
values.
Fixes#6684 by including `dynamicValues` in the
map passed to `EvaluateWithInteractsh()`, so
variables evaluated in earlier requests retain
their values in subsequent requests.
Signed-off-by: Dwi Siswanto <git@dw1.io>
* chore(http): rm early eval in `(*Request).ExecuteWithResults()`
Signed-off-by: Dwi Siswanto <git@dw1.io>
* test: adds variables-threads-previous integration test
Signed-off-by: Dwi Siswanto <git@dw1.io>
* test: adds constants-with-threads integration test
Signed-off-by: Dwi Siswanto <git@dw1.io>
* test: adds race-with-variables integration test
Signed-off-by: Dwi Siswanto <git@dw1.io>
---------
Signed-off-by: Dwi Siswanto <git@dw1.io>
Prev, `FullResponseString()`, `BodyString()`, and
`HeadersString()` were called multiple times per
HTTP response iteration, each call allocating a
new string copy of the response data.
For a 10MB response, this resulted in ~60MB of
redundant string allocs/response (6 calls x 10MB).
Cache the string representations once per `Fill()`
cycle and reuse them throughout the response
processing loop. This reduces allocs from 6 to 3
per response, cutting memory usage by ~50% for
response string handling.
Profiling showed these functions accounting for
~89% of heap allocs (5.7GB out of 6.17GB) during
large scans.
Signed-off-by: Dwi Siswanto <git@dw1.io>
* fix(http): lost request body on retries & redirects
Updates the HTTP protocol to use
`(*retryablehttp.Request).SetBodyString` instead
of direct `Body` assignment.
This fixes#6665 where the request body was
dropped during retries or 307/308 redirects
because `GetBody` was not being populated.
Thanks to @zzyjsj for reporting the bug in the
upstream dependency and the hints!
Signed-off-by: Dwi Siswanto <git@dw1.io>
* empty: add co-author
Co-authored-by: zzy <zzyjsj@users.noreply.github.com>
Signed-off-by: Dwi Siswanto <git@dw1.io>
---------
Signed-off-by: Dwi Siswanto <git@dw1.io>
Co-authored-by: zzy <zzyjsj@users.noreply.github.com>
* fix(interactsh): skip DNS lookups on interactsh domains
to prevent false positives.
Prevents nuclei from resolving interactsh domains
injected in Host headers, which would cause
self-interactions to be incorrectly reported as
matches.
Changes:
* Add `GetHostname()` method to `interactsh.Client`
to expose active server domain.
* Skip CNAME DNS lookups in
`(*http.Request).addCNameIfAvailable` when
hostname matches the
`(*interactsh.Client).GetHostname`.
Fixes#6613
Signed-off-by: Dwi Siswanto <git@dw1.io>
* fix(http): prevent false `interactshDomain` matches
Signed-off-by: Dwi Siswanto <git@dw1.io>
---------
Signed-off-by: Dwi Siswanto <git@dw1.io>
Previously, when using `unsafe: true` with full
URLs (e.g., `GET http://example.com/path HTTP/1.1`),
the `Parse` func would treat the full URL as a
relative path, resulting in malformed requests
like `GET /http://example.com/path HTTP/1.1`.
This occurred because the full URL handling
logic was only executed for non-unsafe requests,
causing unsafe requests with full URLs to fall
through to the unsafe case which wasn't designed
to handle them.
Changes:
* Extract full URL handling before mode-specific
logic runs.
* Convert full URLs to relative paths for both
safe and unsafe modes.
* Update `UnsafeRawBytes` with the correct
relative path when unsafe is true.
* Ensure path merging works correctly with
`disable-path-automerge`.
This fix maintains backward compatibility while
properly supporting the previously broken
combination of unsafe mode with full URLs.
Fixes#6558.
Signed-off-by: Dwi Siswanto <git@dw1.io>
across multiple layers
Fixes timeout configuration conflicts where HTTP
requests would timeout prematurely despite
configured values in `@timeout` annotations or
`-timeout` flags.
RCA:
* `retryablehttp` pkg overriding with default
30s timeout.
* Custom timeouts not propagating to
`retryablehttp` layer.
* Multiple timeout layers not sync properly.
Changes:
* Propagate custom timeouts from `@timeout`
annotations to `retryablehttp` layer.
* Adjust 5-minute maximum cap to prevent DoS via
extremely large timeouts.
* Ensure `retryableHttpOptions.Timeout` respects
`ResponseHeaderTimeout`.
* Add comprehensive tests for timeout capping
behavior.
This allows templates to override global timeout
via `@timeout` annotations while preventing abuse
thru unreasonably large timeout values.
Fixes#6560.
Signed-off-by: Dwi Siswanto <git@dw1.io>
* Enhance matcher compilation with caching for regex and DSL expressions to improve performance. Update template parsing to conditionally retain raw templates based on size constraints.
* Implement caching for regex and DSL expressions in extractors and matchers to enhance performance. Introduce a buffer pool in raw requests to reduce memory allocations. Update template cache management for improved efficiency.
* feat: improve concurrency to be bound
* refactor: replace fmt.Sprintf with fmt.Fprintf for improved performance in header handling
* feat: add regex matching tests and benchmarks for performance evaluation
* feat: add prefix check in regex extraction to optimize matching process
* feat: implement regex caching mechanism to enhance performance in extractors and matchers, along with tests and benchmarks for validation
* feat: add unit tests for template execution in the core engine, enhancing test coverage and reliability
* feat: enhance error handling in template execution and improve regex caching logic for better performance
* Implement caching for regex and DSL expressions in the cache package, replacing previous sync.Map usage. Add unit tests for cache functionality, including eviction by capacity and retrieval of cached items. Update extractors and matchers to utilize the new cache system for improved performance and memory efficiency.
* Add tests for SetCapacities in cache package to ensure cache behavior on capacity changes
- Implemented TestSetCapacities_NoRebuildOnZero to verify that setting capacities to zero does not clear existing caches.
- Added TestSetCapacities_BeforeFirstUse to confirm that initial cache settings are respected and not overridden by subsequent capacity changes.
* Refactor matchers and update load test generator to use io package
- Removed maxRegexScanBytes constant from match.go.
- Replaced ioutil with io package in load_test.go for NopCloser usage.
- Restored TestValidate_AllowsInlineMultiline in load_test.go to ensure inline validation functionality.
* Add cancellation support in template execution and enhance test coverage
- Updated executeTemplateWithTargets to respect context cancellation.
- Introduced fakeTargetProvider and slowExecuter for testing.
- Added Test_executeTemplateWithTargets_RespectsCancellation to validate cancellation behavior during template execution.
* fix: remove undefined errorutil.ShowStackTrace
* feat: add make lint support and integrate with test
* refactor: migrate errorutil to errkit across codebase
- Replace deprecated errorutil with modern errkit
- Convert error declarations from var to func for better compatibility
- Fix all SA1019 deprecation warnings
- Maintain error chain support and stack traces
* fix: improve DNS test reliability using Google DNS
- Configure test to use Google DNS (8.8.8.8) for stability
- Fix nil pointer issue in DNS client initialization
- Keep production defaults unchanged
* fixing logic
* removing unwanted branches in makefile
---------
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
* Use proxy for dns and ssl templates
- while using template execute level function we need to override custom dialer
* rename overridedialer to customdialer
* Add proxy into hash
- proxy client is shared between non proxy requests
* add dialer into request object
- use request.dialer instead of global variable
* resolve comments
* rename dialer
* feat: fixed max-host-error blocking wrong port for template with error
* feat: log total results with time taken at end of execution
* bugfix: skip non-executed requests with progress in flow protocol
* feat: fixed request calculation in http protocol for progress
* misc adjustments
---------
Co-authored-by: Ice3man <nizamulrana@gmail.com>
* Move proxy variable from global to options
- Provides ability to pass diff proxy in single nuclei instance using sdk
* add type check (resolve comments)
* feat: added fuzzing output enhancements
* changes as requested
* misc
* feat: added dfp flag to display fuzz points + misc additions
* feat: added support for fuzzing nested path segments
* feat: added parts to fuzzing requests
* feat: added tracking for parameter occurence frequency in fuzzing
* added cli flag for fuzz frequency
* fixed broken tests
* fixed path based sqli integration test
* feat: added configurable fuzzing aggression level for payloads
* fixed failing test
* feat: added analyzers implementation for fuzzing
* feat: misc changes to analyzer
* feat: misc additions of units + tests fix
* misc changes to implementation
