mirror of
https://github.com/projectdiscovery/nuclei.git
synced 2026-01-31 15:53:10 +08:00
bb79a061bc
`MergeMaps` accounts for 11.41% of allocs (13.8 GB) in clusterbomb mode. With 1,305 combinations per target, this function is called millions of times in the hot path. RCA: * Request generator calls `MergeMaps` with single arg on every payload combination, incurring variadic overhead. * Build request merges same maps multiple times per request. * `BuildPayloadFromOptions` recomputes static CLI options on every call. * Variables calls `MergeMaps` $$2×N$$ times per variable evaluation (once in loop, once in `evaluateVariableValue`) Changes: Core optimizations in maps.go: * Pre-size merged map to avoid rehashing (30-40% reduction) * Add `CopyMap` for efficient single-map copy without variadic overhead. * Add `MergeMapsInto` for in-place mutation when caller owns destination. Hot path fixes: * Replace `MergeMaps(r.currentPayloads)` with `CopyMap(r.currentPayloads)` to eliminates allocation on every combination iteration. * Pre-allocate combined map once, extend in-place during `ForEach` loop instead of creating new map per variable (eliminates $$2×N$$ allocations per request). Caching with concurrency safety: * Cache `BuildPayloadFromOptions` computation in `sync.Map` keyed by `types.Options` ptr, but return copy to prevent concurrent modification. * Cost: shallow copy of ~10-20 entries vs. full merge of vars + env (85-90% savings in typical case) * Clear cache in `closeInternal()` to prevent memory leaks when SDK instances are created or destroyed. Estimated impact: 40-60% reduction in `MergeMaps` allocations (5.5-8.3 GB savings from original 13.8 GB). Safe for concurrent execution and SDK usage with multiple instances. Signed-off-by: Dwi Siswanto <git@dw1.io>
Using Nuclei as Library
Nuclei was primarily built as a CLI tool, but with increasing choice of users wanting to use nuclei as library in their own automation, we have added a simplified Library/SDK of nuclei in v3
Installation
To add nuclei as a library to your go project, you can use the following command:
go get -u github.com/projectdiscovery/nuclei/v3/lib
Or add below import to your go file and let IDE handle the rest:
import nuclei "github.com/projectdiscovery/nuclei/v3/lib"
Basic Example of using Nuclei Library/SDK
// create nuclei engine with options
ne, err := nuclei.NewNucleiEngine(
nuclei.WithTemplateFilters(nuclei.TemplateFilters{Severity: "critical"}), // run critical severity templates only
)
if err != nil {
panic(err)
}
// load targets and optionally probe non http/https targets
ne.LoadTargets([]string{"scanme.sh"}, false)
err = ne.ExecuteWithCallback(nil)
if err != nil {
panic(err)
}
defer ne.Close()
Advanced Example of using Nuclei Library/SDK
For Various use cases like batching etc. you might want to run nuclei in goroutines this can be done by using nuclei.NewThreadSafeNucleiEngine
// create nuclei engine with options
ne, err := nuclei.NewThreadSafeNucleiEngine()
if err != nil{
panic(err)
}
// setup waitgroup to handle concurrency
wg := &sync.WaitGroup{}
// scan 1 = run dns templates on scanme.sh
wg.Add(1)
go func() {
defer wg.Done()
err = ne.ExecuteNucleiWithOpts([]string{"scanme.sh"}, nuclei.WithTemplateFilters(nuclei.TemplateFilters{ProtocolTypes: "http"}))
if err != nil {
panic(err)
}
}()
// scan 2 = run http templates on honey.scanme.sh
wg.Add(1)
go func() {
defer wg.Done()
err = ne.ExecuteNucleiWithOpts([]string{"honey.scanme.sh"}, nuclei.WithTemplateFilters(nuclei.TemplateFilters{ProtocolTypes: "dns"}))
if err != nil {
panic(err)
}
}()
// wait for all scans to finish
wg.Wait()
defer ne.Close()
More Documentation
For complete documentation of nuclei library, please refer to godoc which contains all available options and methods.
Note
| ❗ Disclaimer |
|---|
| This project is in active development. Expect breaking changes with releases. Review the release changelog before updating. |
| This project was primarily built to be used as a standalone CLI tool. Running nuclei as a service may pose security risks. It's recommended to use with caution and additional security measures. |