minor: Validate batch_size configuration when setting it#23054
minor: Validate batch_size configuration when setting it#230542010YOUY01 wants to merge 3 commits into
batch_size configuration when setting it#23054Conversation
| pub const fn new(value: usize) -> Self { | ||
| match NonZeroUsize::new(value) { | ||
| Some(value) => Self(value), | ||
| None => panic!("value must be greater than 0"), |
There was a problem hiding this comment.
lets return config_err it would super confusing if client crashes
There was a problem hiding this comment.
The default constructor for configurations is wrapped inside config_namespace! macro, and it can't handle ? operator on result, so we have to panic here.
But I agree this unsafe API should not be exposed to the caller, so in b37fe74, a private helper is added for unsafe construction, its input is guaranteed to be always valid, since it's only for the hard-coded default value, outside caller can only use the safe try_new()
|
|
||
| /// Creates a [`ConfigNonZeroUsize`], returning a configuration error if | ||
| /// `value` is zero. | ||
| pub fn try_new(value: usize) -> Result<Self> { |
There was a problem hiding this comment.
maybe new with Result<Self> is more obvious choice?
There was a problem hiding this comment.
See above comment.
comphead
left a comment
There was a problem hiding this comment.
Thanks @2010YOUY01 just some minors
Looks like it's a false positive due to the flaky Github CI runner. |
Which issue does this PR close?
Rationale for this change
In datafusion-cli:
A '0' batch_size is a invalid config value, and it's widely used in most operators to control behavior. If operators don't validate
batch_size, the 0 value might cause bugs/panics. This PR rejects it when setting the config.I triggered a bug on my first attempt:
What changes are included in this PR?
Implement a custom type for non-zero usize configuration value on
batch_size, and rejects '0' value on initialization. Rust nativeNonZero<u32>is not used, because custom type allows to provide better error message on errors, and the implementation complexity is similar.Note other invalid inputs like '-1' 'a' are already rejected and covered by existing tests.
Are these changes tested?
Yes, sqllogictest
Are there any user-facing changes?
No