`MergeMaps` accounts for 11.41% of allocs (13.8
GB) in clusterbomb mode. With 1,305 combinations
per target, this function is called millions of
times in the hot path.
RCA:
* Request generator calls `MergeMaps` with single
arg on every payload combination, incurring
variadic overhead.
* Build request merges same maps multiple times
per request.
* `BuildPayloadFromOptions` recomputes static CLI
options on every call.
* Variables calls `MergeMaps` $$2×N$$ times per
variable evaluation (once in loop, once in
`evaluateVariableValue`)
Changes:
Core optimizations in maps.go:
* Pre-size merged map to avoid rehashing (30-40%
reduction)
* Add `CopyMap` for efficient single-map copy
without variadic overhead.
* Add `MergeMapsInto` for in-place mutation when
caller owns destination.
Hot path fixes:
* Replace `MergeMaps(r.currentPayloads)` with
`CopyMap(r.currentPayloads)` to eliminates
allocation on every combination iteration.
* Pre-allocate combined map once, extend in-place
during `ForEach` loop instead of creating new
map per variable (eliminates $$2×N$$ allocations
per request).
Caching with concurrency safety:
* Cache `BuildPayloadFromOptions` computation in
`sync.Map` keyed by `types.Options` ptr, but
return copy to prevent concurrent modification.
* Cost: shallow copy of ~10-20 entries vs. full
merge of vars + env (85-90% savings in typical
case)
* Clear cache in `closeInternal()` to prevent
memory leaks when SDK instances are created or
destroyed.
Estimated impact: 40-60% reduction in `MergeMaps`
allocations (5.5-8.3 GB savings from original
13.8 GB). Safe for concurrent execution and SDK
usage with multiple instances.
Signed-off-by: Dwi Siswanto <git@dw1.io>
* fix(lib): segfault when init engine with `EnableHeadlessWithOpts`
The panic was caused by attempting to log a
sandbox warning before the logger was initialized.
RCA:
* SDK option funcs were exec'd before logger init.
* `EnableHeadlessWithOpts()` attempted to create
browser instance & log warnings during the
config phase.
* `Logger` was only init'd later in `init()`
phase.
* This caused nil pointer dereference when
`MustDisableSandbox()` returned true (root on
Linux/Unix or Windows).
Changes:
* Init `Logger` in `types.DefaultOptions()` to
ensure it's always available before any option
functions execute.
* Init `Logger` field in both
`NewNucleiEngineCtx()` and
`NewThreadSafeNucleiEngineCtx()` from
`defaultOptions.Logger`.
* Move browser instance creation from
`EnableHeadlessWithOpts()` to the `init()` phase
where `Logger` is guaranteed to be available.
* Simplify logger sync logic in `init()` to only
update if changed by `WithLogger` option.
* Add test case to verify headless initialization
works without panic.
The fix maintains backward compatibility while
make sure the logger is always available when
needed by any SDK option function.
Fixes#6601.
Signed-off-by: Dwi Siswanto <git@dw1.io>
* build(make): adds `-timeout 30m -count 1` GOFLAGS in `test` cmd
Signed-off-by: Dwi Siswanto <git@dw1.io>
* Revert "fix(lib): segfault when init engine with `EnableHeadlessWithOpts`"
let see if this pass flaky test.
This reverts commit 63fcb6a1cbe7a4db7a78be766affc70eb237e57e.
* test(engine): let see if this pass flaky test
Signed-off-by: Dwi Siswanto <git@dw1.io>
* Revert "Revert "fix(lib): segfault when init engine with `EnableHeadlessWithOpts`""
This reverts commit 62b4223803ccb1e93593e2e08e39923d76aa20b1.
* test(engine): increase `TestActionNavigate` timeout
Signed-off-by: Dwi Siswanto <git@dw1.io>
* Revert "test(engine): let see if this pass flaky test"
This reverts commit d27cd985cff1b06aa1965ea11f8aa32f00778ab5.
---------
Signed-off-by: Dwi Siswanto <git@dw1.io>
* fix: remove undefined errorutil.ShowStackTrace
* feat: add make lint support and integrate with test
* refactor: migrate errorutil to errkit across codebase
- Replace deprecated errorutil with modern errkit
- Convert error declarations from var to func for better compatibility
- Fix all SA1019 deprecation warnings
- Maintain error chain support and stack traces
* fix: improve DNS test reliability using Google DNS
- Configure test to use Google DNS (8.8.8.8) for stability
- Fix nil pointer issue in DNS client initialization
- Keep production defaults unchanged
* fixing logic
* removing unwanted branches in makefile
---------
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
* Move proxy variable from global to options
- Provides ability to pass diff proxy in single nuclei instance using sdk
* add type check (resolve comments)
* Use the `templateFS` if it's there when calling `OpenFile`
* Add a new constructor
* More refactoring
* Both of my use cases are working
* Fix for legacy assumptions
* minor update: remove gologger debug stmts
---------
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
