feat!(io): Implement Storage for OpenDal #2080
Conversation
| Ok(opendal::Reader::read(self, range).await?.to_bytes()) | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
Moved this to storage.rs, and will move this to iceberg-storage/opendal/ in the future, because this is opendal specific
| let _ = opendal::Writer::close(self).await?; | ||
| Ok(()) | ||
| } | ||
| } |
There was a problem hiding this comment.
Same here, moved this to storage.rs, and will move this to iceberg-storage/opendal/ in the future, because this is opendal specific
| async fn close(&mut self) -> crate::Result<()> { | ||
| self.as_mut().close().await | ||
| } | ||
| } |
There was a problem hiding this comment.
Deleted this because we pass Box<dyn FileWrite> around and use boxed items directly.
I updated AsyncFileWriter accordingly
| let mut writer = self.writer().await?; | ||
| writer.write(bs).await?; | ||
| writer.close().await | ||
| self.storage.write(&self.path, bs).await |
There was a problem hiding this comment.
I think there is a minor behavior difference here. The new implementation calls Operator::write directly while the old one uses Writer::write and then explicitly closes it.
According to OpenDal's doc, the difference seems trivial
| format!("{relative_path}/") | ||
| }; | ||
| Ok(op.remove_all(&path).await?) | ||
| pub async fn delete_prefix(&self, path: impl AsRef<str>) -> Result<()> { |
There was a problem hiding this comment.
This is a breaking change, the method is renamed to delete_prefix
CTTY
changed the title
1 participant
Which issue does this PR close?
What changes are included in this PR?
Wired
OpenDalwithStoragetrait.FileIOstill uses hardcodedOpenDalas of now so no customer facing behavior changesAre these changes tested?
No new tests added, rely on existing tests