* fix(http): interactsh matching with `payloads`
in parallel execution.
Templates using `payloads` with Interactsh
matchers failed to detect OAST interactions
because the parallel HTTP execution path (used
when `payloads` are present) did not register
Interactsh request events, unlike the seq path.
This caused incoming interactions to lack
associated request context, preventing matchers
from running and resulting in missed detections.
Fix#5485 by wiring
`(*interactsh.Client).RequestEvent` registration
into the parallel worker goroutine, make sure both
execution paths handle Interactsh correlation
equally.
Signed-off-by: Dwi Siswanto <git@dw1.io>
* test: add interactsh with `payloads` integration
Signed-off-by: Dwi Siswanto <git@dw1.io>
* test: disable interactsh-with-payloads
Signed-off-by: Dwi Siswanto <git@dw1.io>
---------
Signed-off-by: Dwi Siswanto <git@dw1.io>
* adding min auth support
* adding unauth list modules + auth list files in module
* example
* adding rsync test
* bump go.mod
---------
Co-authored-by: Dwi Siswanto <git@dw1.io>
Make sure postgres Exec/ExecContext are invoked with the correct
argument order, preventing context from being passed as the query.
* fixing pg syntax
* adding test
Restore backwards compat for JavaScript protocol
templates that omit the `Port` argument.
Regression was introduced in f4f2e9f2, which
removed the fallback for empty `Port` in
`(*Request).ExecuteWithResults`, causing templates
without `Port` to be silently skipped.
Now, if no `Port` is specified, the engine
executes the JavaScript block using the target
URL's port.
Fixes#6708.
Signed-off-by: Dwi Siswanto <git@dw1.io>
* fix(http): pass `dynamicValues` to `EvaluateWithInteractsh`
When `LazyEval` is true (triggered by `variables`
containing `BaseURL`, `Hostname`,
`interactsh-url`, etc.), variable expressions are not
eval'ed during YAML parsing & remain as raw exprs
like "{{rand_base(5)}}".
At request build time, `EvaluateWithInteractsh()`
checks if a variable already has a value in the
passed map before re-evaluating its expression.
But, `dynamicValues` (which contains the template
context with previously eval'ed values) was not
being passed, causing exprs like `rand_*` to be
re-evaluated on each request, producing different
values.
Fixes#6684 by including `dynamicValues` in the
map passed to `EvaluateWithInteractsh()`, so
variables evaluated in earlier requests retain
their values in subsequent requests.
Signed-off-by: Dwi Siswanto <git@dw1.io>
* chore(http): rm early eval in `(*Request).ExecuteWithResults()`
Signed-off-by: Dwi Siswanto <git@dw1.io>
* test: adds variables-threads-previous integration test
Signed-off-by: Dwi Siswanto <git@dw1.io>
* test: adds constants-with-threads integration test
Signed-off-by: Dwi Siswanto <git@dw1.io>
* test: adds race-with-variables integration test
Signed-off-by: Dwi Siswanto <git@dw1.io>
---------
Signed-off-by: Dwi Siswanto <git@dw1.io>
* Add ai flag
* Add AI flag 2
* fix stdin
* fix stdin 2
* minor
* print both url and path
* store ai generated templates in `$HOME/nuclei-templates/pdcp`
* todo
* do not remove all
* make it less restrictive
* use retryablehttp
* fix creds check
* return errs
* return more detailed err for non-ok status code
* add prompt validation
* fix integration tests
---------
Co-authored-by: Doğan Can Bakır <dogancanbakir@protonmail.com>
* feat: added fuzzing output enhancements
* changes as requested
* misc
* feat: added dfp flag to display fuzz points + misc additions
* feat: added support for fuzzing nested path segments
* feat: added parts to fuzzing requests
* feat: added tracking for parameter occurence frequency in fuzzing
* added cli flag for fuzz frequency
* fixed broken tests
* fixed path based sqli integration test
* feat: added configurable fuzzing aggression level for payloads
* fixed failing test
* add default get method
* remove residual payload logic from old implementation
* fuzz: clone current state of component
* fuzz: bug fix stacking of payloads in multiple mode
* improve stdout template loading stats
* stdout: force display warnings if no templates are loaded
* update flags in README.md
* quote non-ascii chars in extractor output
* aws request signature can only be used in signed & verified tmpls
* deprecate request signature
* remove logic related to deprecated fuzzing input
* update test to use ordered params
* fix interactsh-url lazy eval: #4946
* output: skip unnecessary updates when unescaping
* updates as per requested changes
* feat: move fuzz package to root directory
* feat: added support for input providers like openapi,postman,etc
* feat: integration of new fuzzing logic in engine
* bugfix: use and instead of or
* fixed lint errors
* go mod tidy
* add new reqresp type + bump utils
* custom http request parser
* use new struct type RequestResponse
* introduce unified input/target provider
* abstract input formats via new inputprovider
* completed input provider refactor
* remove duplicated code
* add sdk method to load targets
* rename component url->path
* add new yaml format + remove duplicated code
* use gopkg.in/yaml.v3 for parsing
* update .gitignore
* refactor/move + docs fuzzing in http protocol
* fuzz: header + query integration test using fuzzplayground
* fix integration test runner in windows
* feat add support for filter in http fuzz
* rewrite header/query integration test with filter
* add replace regex rule
* support kv fuzzing + misc updates
* add path fuzzing example + misc improvements
* fix matchedURL + skip httpx on multi formats
* cookie fuzz integration test
* add json body + params body tests
* feat add multipart/form-data fuzzing support
* add all fuzz body integration test
* misc bug fixes + minor refactor
* add multipart form + body form unit tests
* only run fuzzing templates if -fuzz flag is given
* refactor/move fuzz playground server to pkg
* fix integration test + refactor
* add auth types and strategies
* add file auth provider
* start implementing auth logic in http
* add logic in http protocol
* static auth implemented for http
* default :80,:443 normalization
* feat: dynamic auth init
* feat: dynamic auth using templates
* validate targets count in openapi+swagger
* inputformats: add support to accept variables
* fix workflow integration test
* update lazy cred fetch logic
* fix unit test
* drop postman support
* domain related normalization
* update secrets.yaml file format + misc updates
* add auth prefetch option
* remove old secret files
* add fuzzing+auth related sdk options
* fix/support multiple mode in kv header fuzzing
* rename 'headers' -> 'header' in fuzzing rules
* fix deadlock due to merge conflict resolution
* misc update
* add bool type in parsed value
* add openapi validation+override+ new flags
* misc updates
* remove optional path parameters when unavailable
* fix swagger.yaml file
* misc updates
* update print msg
* multiple openapi validation enchancements + appMode
* add optional params in required_openapi_vars.yaml file
* improve warning/verbose msgs in format
* fix skip-format-validation not working
* use 'params/parameter' instead of 'variable' in openapi
* add retry support for falky tests
* fix nuclei loading ignored templates (#4849)
* fix tag include logic
* fix unit test
* remove quoting in extractor output
* remove quote in debug code command
* feat: issue tracker URLs in JSON + misc fixes (#4855)
* feat: issue tracker URLs in JSON + misc fixes
* misc changes
* feat: status update support for issues
* feat: report metadata generation hook support
* feat: added CLI summary of tickets created
* misc changes
* introduce `disable-unsigned-templates` flag (#4820)
* introduce `disable-unsigned-templates` flag
* minor
* skip instead of exit
* remove duplicate imports
* use stats package + misc enhancements
* force display warning + adjust skipped stats in unsigned count
* include unsigned skipped templates without -dut flag
---------
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
* Purge cache on global callback set (#4840)
* purge cache on global callback set
* lint
* purging cache
* purge cache in runner after loading templates
* include internal cache from parsers + add global cache register/purge via config
* remove disable cache purge option
---------
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
* misc update
* add application/octet-stream support
* openapi: support path specific params
* misc option + readme update
---------
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
Co-authored-by: Tarun Koyalwar <45962551+tarunKoyalwar@users.noreply.github.com>
Co-authored-by: Dogan Can Bakir <65292895+dogancanbakir@users.noreply.github.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
* handle 1 more edgecase
* add integration test for this edgecase
* fix multi-http-var-sharing with integration test
* add -payload-concurrency (-pc) flag
* fix missing internal:true login in multiprotocol engine
* fix/handle absolute invalid url parsing
* support -pc & -jc in go sdk
* fix missing variables in code protocol operators
* add payload count parallelhttp check
* feat http response memory optimization + reuse buffers
* update nuclei version
* feat: reuse js vm's and compile to programs
* fix failing http integration test
* remove dead code + add -jsc
* feat reuse js vms in pool with concurrency
* update comments as per review
* bug fix+ update interactsh test to look for dns interaction
* try enabling all interactsh integration tests
---------
Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
* log warnings + use scanContext in flow
* refactor flow to use scanContext + log all events
* feat: internal matcher
* fix integration test
* bug fix extractor: merge dynamic values, fix missing extractors in file
* flow: fix 'No Results Found' if last statement output is false
* fix unit test
* fix race-condition & oow in extracted file output
* add mutex for file.Write + set finalizer for os.File
* fix integration test
* disable extractor save to file in lib mode(configurable)
* use sync.Once for init
* disable out of bound image write in headless
* misc updates
* fix headless screenshot test
* fix extractor save to file integration test
* remove 'to' feature in extractors
* fix net read
* only return N bytes if extra available
* use ConnReadN from readerutil
* add integration test
* print unsigned warning in stderr
* fix js protocol in flow #4318
* fix integration test: url encoding issue
* fix network protocol issue + integration tests
* multiple improvements to integration test
* replace all conn.Read() from tests
* disable network-basic.yaml in windows
* disable code protocol in win CI
* fix bitwise login ps1-snippet.yaml
* hide previous matcher events in flow
* remove dead code+ update integration tests
---------
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
* add randstr preprocessor to defaults
* fix indexing in http + preprocessor integration test
* add multi-request integration test
* skip test if asnmap is down
