CAMEL-23594: Fix property resolution issues in kamelet endpoint URIs#23391
CAMEL-23594: Fix property resolution issues in kamelet endpoint URIs#23391cunningt wants to merge 1 commit into
Conversation
Two related fixes for property values used in kamelet endpoint URIs: 1. Trim whitespace from resolved property values in DefaultPropertiesParser. Property values containing trailing whitespace (e.g., from YAML literal block scalars using "|") cause URISyntaxException when resolved into endpoint URIs. The trim is safe since legitimate property values never depend on surrounding whitespace. 2. URL-decode kamelet endpoint parameters in KameletComponent. When YAML DSL creates kamelet URIs, it URL-encodes property values (e.g., "application/json" → "application%2Fjson"). Since KameletComponent uses useRawUri=true to preserve sensitive values, automatic URL-decoding is skipped. This fix manually decodes non-RAW parameter values so they are passed correctly to kamelet templates. Together these fixes resolve kamelet integration test failures where property values were either malformed (trailing newline causing URISyntaxException) or incorrectly encoded (Content-Type header sent as "application%2Fjson" instead of "application/json"). Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
🌟 Thank you for your contribution to the Apache Camel project! 🌟 🐫 Apache Camel Committers, please review the following items:
|
|
🧪 CI tested the following changed modules:
Build reactor — dependencies compiled but only changed modules were tested (2 modules)
|
davsclaus
left a comment
There was a problem hiding this comment.
Claude Code on behalf of Claus Ibsen
Thanks for tackling this issue — the URL-encoding problem in kamelet parameters is a real pain point. I have a few concerns with the current approach that I think are worth addressing before merging.
Summary
-
DefaultPropertiesParser.trim()is too broad — This affects every property resolution across the entire framework, not just YAML/kamelet scenarios. Property values that legitimately contain leading/trailing whitespace (passwords, PEM content, regex patterns, template strings) would be silently corrupted. The root cause — YAML literal block scalars (|) preserving trailing newlines — should be fixed at the YAML parsing layer, not in the global properties parser. -
No tests included — Per project guidelines, every PR must include tests for bug fixes. At minimum: a test showing URL-encoded kamelet parameter values are properly decoded, a test for the properties parser whitespace handling, and a regression test ensuring RAW-wrapped values are not double-decoded.
-
URL-decoding may affect non-YAML code paths —
createEndpoint()is called from all DSLs (Java, XML, YAML). If a value arrives without encoding (e.g., from Java DSL),URLDecoder.decode()could corrupt values containing%characters (e.g.,100%would trigger decode errors). The fix should either be guarded to YAML-originated calls, or the encoding should be fixed at the source (YAML DSL query string construction) rather than decoded at the destination. -
Missing upgrade guide entry — The
trim()change inDefaultPropertiesParseris a behavior change to a core component that should be documented.
This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.
| answer = value; | ||
| } | ||
| return answer; | ||
| return answer != null ? answer.trim() : null; |
There was a problem hiding this comment.
This .trim() applies to every property resolution across the entire framework, not just YAML/kamelet scenarios. Property values that legitimately contain leading/trailing whitespace (passwords with trailing spaces, PEM data, regex patterns) would be silently corrupted.
The root cause is YAML literal block scalars (|) preserving trailing newlines. This should be fixed at the YAML parsing layer where the newline is introduced, not in the global properties parser.
Consider removing this change and instead trimming at the YAML DSL level where property values are extracted from block scalars.
| if (entry.getValue() instanceof String s | ||
| && !s.startsWith(URISupport.RAW_TOKEN_PREFIX + "(") | ||
| && !s.startsWith(URISupport.RAW_TOKEN_PREFIX + "{")) { | ||
| entry.setValue(URLDecoder.decode(s, StandardCharsets.UTF_8)); |
There was a problem hiding this comment.
createEndpoint() is called from all DSLs (Java, XML, YAML), not just YAML. If a value arrives without URL-encoding (e.g., from Java DSL to("kamelet:foo?contentType=application/json")), URLDecoder.decode() will still process it. While application/json survives decoding, a value containing % (like 100%done) would be corrupted or throw an IllegalArgumentException.
Consider either:
- Fixing the encoding at the source (the YAML DSL's
URISupport.createQueryString()call) so values aren't double-encoded - Or guarding this decode with a check that the value actually contains URL-encoded sequences (e.g.,
%xxpatterns)
|
I think the double decoding is a bit weird. Also can you point to which kamelts yaml files are having problems. It can be that they do wrong yaml - when you have multi line its |= vs >= etc |
gnodet
left a comment
There was a problem hiding this comment.
Thanks for working on this, Tom. The URL-encoding asymmetry between useRawUri()=true and the YAML DSL's createQueryString(params) is a real problem. I have two concerns with the current approach -- one blocking (the trim() change) and one that I think needs discussion (the URL-decode fix).
1. DefaultPropertiesParser.trim() -- too broad, risk of data corruption (blocking)
This change applies to every property resolution across the entire Camel framework, not just kamelet/YAML scenarios. Properties resolved via {{...}} placeholders are used for passwords, PEM certificates, regex patterns, file paths, and many other values where leading/trailing whitespace could be significant.
The root cause is that YAML literal block scalars (|) preserve trailing newlines. The right fix should be at the YAML parsing layer where the values are extracted, not in the global properties parser. The DefaultPropertiesParser has been stable for years and changing it silently could break existing users who depend on whitespace-preserving property values (e.g., System.getProperty() or environment variables that intentionally contain whitespace).
I would recommend removing this part of the change entirely, and if trailing newlines from YAML block scalars are a problem, fix it in the YAML DSL property extraction logic.
2. URL-decoding in KameletComponent.createEndpoint() -- fragile
The URL-decode fix works around the encoding done by URISupport.createQueryString(params) (which uses encode=true by default) in YamlRoutesBuilderLoader line 605. However:
-
createEndpoint()is called from ALL DSLs (Java, XML, YAML), not just YAML. When a Java DSL route usesto("kamelet:foo?contentType=application/json"), the valueapplication/jsonarrives unencoded. WhileURLDecoder.decode()is a no-op for that specific value, any value containing a bare%(like100%completeor a regex[a-z]%[0-9]) will throwIllegalArgumentException. -
The fix in CAMEL-23284 took the opposite approach for
{{placeholder}}encoding: it un-encoded at the source (query.replace("%7B%7B", "{{").replace("%7D%7D", "}}")inYamlRoutesBuilderLoader). Fixing the encoding at the source would be more consistent and safer -- e.g., usingcreateQueryString(params, false)for kamelet URIs inYamlRoutesBuilderLoader, similar to whatKameletDeserializerandYamlSupportalready do. -
If decoding at the destination is preferred for some reason, at minimum wrap the
URLDecoder.decode()call in a try/catch forIllegalArgumentExceptionso it does not crash on values with bare%characters.
3. Missing tests
Per project guidelines, bug-fix PRs must include tests. This PR needs at minimum:
- A test showing URL-encoded kamelet parameter values (e.g.,
application%2Fjson) are properly decoded - A regression test ensuring RAW-wrapped values survive the decode step
- A test for a value containing bare
%(to verify it does not crash)
Claude Code on behalf of Guillaume Nodet
| answer = value; | ||
| } | ||
| return answer; | ||
| return answer != null ? answer.trim() : null; |
There was a problem hiding this comment.
This .trim() applies to every property resolution across the entire Camel framework -- system properties, environment variables, .properties files, Spring Boot config, etc. Property values that legitimately contain trailing whitespace (passwords, PEM data, regex, template strings) would be silently corrupted.
The root cause is YAML literal block scalars (|) preserving trailing newlines. Please fix this at the YAML parsing layer where property values are extracted, not in the global properties parser.
Suggest removing this change entirely.
Claude Code on behalf of Guillaume Nodet
| if (entry.getValue() instanceof String s | ||
| && !s.startsWith(URISupport.RAW_TOKEN_PREFIX + "(") | ||
| && !s.startsWith(URISupport.RAW_TOKEN_PREFIX + "{")) { | ||
| entry.setValue(URLDecoder.decode(s, StandardCharsets.UTF_8)); |
There was a problem hiding this comment.
URLDecoder.decode() will throw IllegalArgumentException on values containing bare % characters (e.g., 100%complete or [a-z]%[0-9]). This createEndpoint() method is called from all DSLs, not just YAML, so non-encoded values will reach here.
Consider either:
- Fix at the source: use
createQueryString(params, false)inYamlRoutesBuilderLoaderfor kamelet URIs (likeKameletDeserializerandYamlSupportalready do withencode=false). This would be consistent with the CAMEL-23284 approach. - Or guard the decode: at minimum wrap in try/catch
IllegalArgumentExceptionand fall back to the original value.
| entry.setValue(URLDecoder.decode(s, StandardCharsets.UTF_8)); | |
| for (Map.Entry<String, Object> entry : parameters.entrySet()) { | |
| if (entry.getValue() instanceof String s | |
| && !s.startsWith(URISupport.RAW_TOKEN_PREFIX + "(") | |
| && !s.startsWith(URISupport.RAW_TOKEN_PREFIX + "{")) { | |
| try { | |
| entry.setValue(URLDecoder.decode(s, StandardCharsets.UTF_8)); | |
| } catch (IllegalArgumentException e) { | |
| // value contains bare % not part of URL encoding, leave as-is | |
| } | |
| } | |
| } |
Though option 1 (fixing the encoding at the YAML DSL source) would be the cleanest solution.
Claude Code on behalf of Guillaume Nodet
|
Thank you for the feedback! I'm working on changes to minimize the scope of where trim / decode run and target kamelets. The issue here I see is that there are 7 transformation kamelet integration tests failing : https://github.com/apache/camel-kamelets/actions/runs/26155039895/job/76932071431
The root cause of all of the failures there is that the content type is being URL-encoded and application/json → application%2Fjson, causing unexpected behavior and lots of test failures. |
Problem
Kamelet integration tests were failing due to two property resolution issues:
Trailing whitespace in property values — YAML literal block scalars (
|) preserve trailing newlines, which causeURISyntaxExceptionwhen the property value is resolved into an endpoint URI string.URL-encoded property values — The YAML DSL uses
URISupport.createQueryString()to build kamelet URIs, which URL-encodes property values (e.g.,application/json→application%2Fjson). SinceKameletComponentusesuseRawUri()=trueto preserve sensitive values, automatic URL-decoding during URI parsing is skipped, leaving encoded values likeapplication%2Fjsonto be passed to kamelet templates.Solution
Fix 1: Trim whitespace in DefaultPropertiesParser
File:
core/camel-base/src/main/java/org/apache/camel/component/properties/DefaultPropertiesParser.javaAdded
.trim()when returning resolved property values indoGetPropertyValue(). This strips leading/trailing whitespace, which is safe since no legitimate property value depends on surrounding whitespace.Fix 2: URL-decode kamelet parameters
File:
components/camel-kamelet/src/main/java/org/apache/camel/component/kamelet/KameletComponent.javaAdded URL-decoding for non-RAW string parameter values in
createEndpoint(), between extraction and RAW resolution. This reverses the encoding done byURISupport.createQueryString()while preserving RAW-wrapped sensitive values (which useRAW()syntax and skip URL encoding).Testing
org.apache.camel.component.properties.*Test)These fixes resolve kamelet integration test failures where:
URISyntaxExceptionapplication%2Fjsonwere incorrectly passed asContent-Typeheaders, causing HTTP 415 errorsRelated Issues
CAMEL-23594