Commit Graph

648 Commits

Author SHA1 Message Date
Doğan Can Bakır
cc1d646347 log failed expr compilations 2025-10-13 14:41:27 +03:00
Mzack9999
4e2af6bf67 Merge pull request #6373 from mielverkerken/dev
Add option to control number of concurrent templates loaded on startup
2025-10-09 16:05:39 +02:00
Mzack9999
e0d3bb45ed small changes 2025-10-06 22:38:43 +02:00
Matej Smycka
67571b3cfe feat: http(s) probing optimization 2025-10-06 13:30:02 +02:00
Mzack9999
75016d1e96 Merge pull request #6500 from projectdiscovery/dwisiswant0/fix/issue-6499-6498
fix: suppress warn code flag not found & excludes known misc dir
2025-10-06 11:06:48 +02:00
Doğan Can Bakır
3597ab07f0 ai recommendations 2025-10-01 12:46:43 +03:00
Doğan Can Bakır
0f4f4b94e6 Merge branch 'dev' into mielverkerken-dev 2025-10-01 12:36:55 +03:00
Dwi Siswanto
c903da3a0c fix(config): normalize fpath in IsTemplate
* normalize file `fpath` in `IsTemplate` using
  filepath.FromSlash to ensure consistent matching
  across platforms.
* update `GetKnownMiscDirectories` docs to clarify
  that trailing slashes prevent false positives,
  since `IsTemplate` compares against normalized
  full paths.

Signed-off-by: Dwi Siswanto <git@dw1.io>
2025-09-30 00:40:47 +07:00
Dwi Siswanto
b529125031 refactor(confif): update known misc dirs & improve IsTemplate func
Signed-off-by: Dwi Siswanto <git@dw1.io>
2025-09-27 16:02:12 +07:00
Dwi Siswanto
ca11a2fad6 fix(disk): uses config.IsTemplate instead
fixes #6499

Signed-off-by: Dwi Siswanto <git@dw1.io>
2025-09-27 15:21:38 +07:00
Dwi Siswanto
7d450507f7 feat(config): adds known misc directories
and excludes em in IsTemplate func.

Signed-off-by: Dwi Siswanto <git@dw1.io>
2025-09-27 15:20:45 +07:00
Dwi Siswanto
95a72cfd50 fix(templates): suppress warn code flag not found
on validate.

fixes #6498

Signed-off-by: Dwi Siswanto <git@dw1.io>
2025-09-27 13:34:28 +07:00
Mzack9999
cb2d93174a fixing logic 2025-09-25 22:46:40 +02:00
Mzack9999
61bd0828dc Merge branch 'dev' into RDP-Enc-func 2025-09-25 22:07:17 +02:00
Dogan Can Bakir
d44f07f648 Merge pull request #6495 from projectdiscovery/fix_headless_loading
fix headless template loading logic when `-dast` option is enabled
2025-09-24 13:11:28 +03:00
Nakul Bharti
93be3b8291 fix: improve cleanup in parallel execution (#6490) 2025-09-24 01:12:43 +05:30
Doğan Can Bakır
202524283b fix headless template loading logic when -dast option is enabled 2025-09-23 16:43:08 +03:00
Nakul Bharti
8ea5061f5e jira: hotfix for Cloud to use /rest/api/3/search/jql (#6489)
* jira: hotfix for Cloud to use /rest/api/3/search/jql in FindExistingIssue; add live test verifying v3 endpoint

* jira: fix Cloud v3 search response handling (no total); set Self from base

* fix lint error

* tests(jira): apply De Morgan to satisfy staticcheck QF1001
2025-09-22 22:44:10 +05:30
Dwi Siswanto
d2cf69aebb feat(fuzz): enhance MultiPartForm with metadata APIs (#6486)
* feat(fuzz): enhance `MultiPartForm` with metadata APIs

* add `SetFileMetadata`/`GetFileMetadata` APIs for
  file metadata management.
* implement RFC-2046 boundary validation
  (max 70 chars).
* add boundary validation in `Decode` method.

* fix `filesMetadata` initialization.
* fix mem leak by removing defer from file reading
  loop.
* fix file metadata overwriting by storing first
  file's metadata instead of last.

Closes #6405, #6406

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(fuzz): satisfy lint errs

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
2025-09-22 22:09:24 +05:30
Mzack9999
39e9286371 Feat 6231 deadlock (#6469)
* fixing recursive deadlock

* using atomics

* fixing init
2025-09-22 21:49:56 +05:30
halcyondream
792998d8e2 Refactored header-based auth scans not to normalize the header names. (#6479)
* Refactored header-based auth scans not to normalize the header names.

* Removed the header validation as it's not really useful here.

* adding docs

---------

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2025-09-16 04:35:00 +05:30
Nakul Bharti
c4fa2c74c1 cache, goroutine and unbounded workers management (#6420)
* Enhance matcher compilation with caching for regex and DSL expressions to improve performance. Update template parsing to conditionally retain raw templates based on size constraints.

* Implement caching for regex and DSL expressions in extractors and matchers to enhance performance. Introduce a buffer pool in raw requests to reduce memory allocations. Update template cache management for improved efficiency.

* feat: improve concurrency to be bound

* refactor: replace fmt.Sprintf with fmt.Fprintf for improved performance in header handling

* feat: add regex matching tests and benchmarks for performance evaluation

* feat: add prefix check in regex extraction to optimize matching process

* feat: implement regex caching mechanism to enhance performance in extractors and matchers, along with tests and benchmarks for validation

* feat: add unit tests for template execution in the core engine, enhancing test coverage and reliability

* feat: enhance error handling in template execution and improve regex caching logic for better performance

* Implement caching for regex and DSL expressions in the cache package, replacing previous sync.Map usage. Add unit tests for cache functionality, including eviction by capacity and retrieval of cached items. Update extractors and matchers to utilize the new cache system for improved performance and memory efficiency.

* Add tests for SetCapacities in cache package to ensure cache behavior on capacity changes

- Implemented TestSetCapacities_NoRebuildOnZero to verify that setting capacities to zero does not clear existing caches.
- Added TestSetCapacities_BeforeFirstUse to confirm that initial cache settings are respected and not overridden by subsequent capacity changes.

* Refactor matchers and update load test generator to use io package

- Removed maxRegexScanBytes constant from match.go.
- Replaced ioutil with io package in load_test.go for NopCloser usage.
- Restored TestValidate_AllowsInlineMultiline in load_test.go to ensure inline validation functionality.

* Add cancellation support in template execution and enhance test coverage

- Updated executeTemplateWithTargets to respect context cancellation.
- Introduced fakeTargetProvider and slowExecuter for testing.
- Added Test_executeTemplateWithTargets_RespectsCancellation to validate cancellation behavior during template execution.
2025-09-15 23:48:02 +05:30
Nakul Bharti
d4f1a815ed fix: update go jira deps (#6475)
* fix: handle jira deprecated endpoint

* refactor: update Jira issue search result structure to include 'Self' field

* Revert "refactor: update Jira issue search result structure to include 'Self' field"

This reverts commit b0953419d3.

* Revert "fix: handle jira deprecated endpoint"

This reverts commit 1fc05076cd.

* chore(deps): bump github.com/andygrunwald/go-jira to v1.16.1 and tidy

* fix(jira): migrate Issue.Search to SearchV2JQL with explicit Fields
2025-09-15 18:23:08 +05:30
Mzack9999
876974f38b Merge pull request #6422 from zy9ard3/dev
No changes message for github custom template update to INF from ERR for better logging
2025-09-12 21:21:40 +02:00
nu11z
ca543d7885 Remove the stack trace when the nuclei-ignore file does not exist (#6455)
* remove the stack trace when the nuclei-ignore file does not exist

* removing useless debug stack

---------

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2025-09-12 23:36:36 +05:30
Mzack9999
fde6f72934 refactor 2025-09-12 19:03:56 +02:00
Mzack9999
3af37362e3 Merge pull request #6472 from projectdiscovery/maint-rate-unlimit
centralizing ratelimiter logic
2025-09-12 18:59:13 +02:00
Mzack9999
99a9ce398d Merge branch 'dev' into pr/6422 2025-09-12 18:25:18 +02:00
Mzack9999
48af0b4f6c Merge pull request #6464 from projectdiscovery/jira-custom-template-syntax
feat: added new text/template syntax to jira custom fields
2025-09-12 18:21:29 +02:00
Mzack9999
089e2a4ee0 centralizing ratelimiter logic 2025-09-12 17:46:42 +02:00
Mzack9999
521a21c06a Merge branch 'dev' into feat-4842-vnc 2025-09-12 11:51:17 +02:00
Mzack9999
1acd40f97f Merge pull request #6465 from projectdiscovery/4690_dont_load_dup_templates
dont load templates with the same ID
2025-09-12 11:46:51 +02:00
Mzack9999
c863143771 lint 2025-09-12 10:35:09 +02:00
Mzack9999
5c8da8d88b code from https://github.com/projectdiscovery/nuclei/pull/6427 2025-09-12 10:29:42 +02:00
Mzack9999
c487e59602 lint 2025-09-11 21:41:59 +02:00
Mzack9999
1f8dc4c358 Merge branch 'dev' into pr/6261 2025-09-11 21:33:40 +02:00
Mzack9999
608159bbbe lint 2025-09-10 19:53:23 +02:00
Mzack9999
b05359bc82 using synclockmap 2025-09-10 19:48:36 +02:00
Doğan Can Bakır
4916cf34f0 dont load templates with the same ID 2025-09-10 16:44:12 +03:00
Ice3man
f460bf926d feat: added additional text/template helpers 2025-09-10 17:32:43 +05:30
Ice3man
218a2f69a5 feat: added new text/template syntax to jira custom fields 2025-09-10 16:51:20 +05:30
mkrs2404
9c64a1cb9b Reporting validation (#6456)
* add custom validator for reporting issues

* use httpx dev branch

* remove yaml marshal/unmarshal for validator callback
2025-09-05 19:53:26 +05:30
cui
d76187f99a Refactor to use reflect.TypeFor (#6428) 2025-08-27 22:31:04 +05:30
zy9ard3
1f0aef970c fix for error.Is false return 2025-08-26 10:48:10 +05:30
zy9ard3
5b7debf349 Update pkg/external/customtemplates/github.go
Co-authored-by: Dwi Siswanto <25837540+dwisiswant0@users.noreply.github.com>
2025-08-26 09:05:31 +05:30
Mzack9999
e83382d4e4 lint 2025-08-25 15:33:21 +02:00
Mzack9999
b61321cd19 Merge branch 'dev' into feat-4842-vnc 2025-08-25 15:22:14 +02:00
Mzack9999
f20f95f67e integration test 2025-08-25 15:13:23 +02:00
Mzack9999
efcef55681 lint 2025-08-25 13:59:01 +02:00
Dwi Siswanto
a1b5a0ed99 fix(fuzz): handles duplicate multipart form field names (#6404)
* fix: handle duplicate field names in multipart form encoding

* fix(fuzz): handles `[]any` type in `*MultiPartForm.Encode`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* test(fuzz): adds panic recovery & display encoded out

Signed-off-by: Dwi Siswanto <git@dw1.io>

* fix(fuzz): incorrectly treated mixed type field

in `*MultiPartForm.Encode`

Signed-off-by: Dwi Siswanto <git@dw1.io>

* test(fuzz): refactor compare w decoded instead

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(fuzz): prealloc for `[]any` type

Signed-off-by: Dwi Siswanto <git@dw1.io>

* fix(fuzz): treats nil value as empty string

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(fuzz): rm early error return for non-array file

Signed-off-by: Dwi Siswanto <git@dw1.io>

* test(fuzz): adds `TestMultiPartFormFileUpload` test

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
Co-authored-by: yusei-wy <31252054+yusei-wy@users.noreply.github.com>
2025-08-25 13:42:51 +05:30