feat(bindings/python): Add from_uri constructors for Operators#6993
feat(bindings/python): Add from_uri constructors for Operators#6993clbarnes wants to merge 11 commits intoapache:mainfrom
Conversation
|
Tests pass but I'm not clear how best to test the new functionality as it would mean parsing the config provided by the setup_config fixture and then constructing a URI out of it, which is going to be different for each service. |
|
Thank you for working on this! I will invite @chitralverma to take a review on this. |
|
@clbarnes can you run stubgen with just to get the python stubs as well? |
|
I found that running stubgen locally drops a bunch of services and deletes a lot of generated code as well - possibly all the services which have been split out of core into their own crates? --- a/bindings/python/src/services.rs
+++ b/bindings/python/src/services.rs
@@ -62,16 +62,6 @@ submit! {
#[pyo3(rename_all = "PascalCase")]
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub enum PyScheme {
- #[cfg(feature = "services-aliyun-drive")]
- AliyunDrive,
- #[cfg(feature = "services-alluxio")]
- Alluxio,
- #[cfg(feature = "services-azblob")]
- Azblob,
- #[cfg(feature = "services-azdls")]
- Azdls,
- #[cfg(feature = "services-azfile")]
- Azfile,
#[cfg(feature = "services-b2")]
B2,
#[cfg(feature = "services-cacache")]
@@ -84,24 +74,14 @@ pub enum PyScheme {
Dropbox,
#[cfg(feature = "services-fs")]
Fs,
- #[cfg(feature = "services-ftp")]
- Ftp,
- #[cfg(feature = "services-gcs")]
- Gcs,
#[cfg(feature = "services-gdrive")]
Gdrive,
- #[cfg(feature = "services-ghac")]
- Ghac,
#[cfg(feature = "services-gridfs")]
Gridfs,
- #[cfg(feature = "services-hdfs-native")]
- HdfsNative,
#[cfg(feature = "services-http")]
Http,
#[cfg(feature = "services-huggingface")]
Huggingface,
- #[cfg(feature = "services-ipfs")]
- Ipfs,
#[cfg(feature = "services-ipmfs")]
Ipmfs,
#[cfg(feature = "services-koofr")]etc.. That comes from the The second step doesn't finish successfully - but This could possibly resolved using the devcontainer, but that has issues too: #6992 |
|
Hi @clbarnes, this PR has been quiet for ~4 months. Is the implementation still in progress, or would you welcome help getting it over the line? Happy to assist with rebase/CI fixes or take it over with attribution if that would help. |
|
I will rebase and give it another try, but I was blocked last time with setting up an environment with everything in place for getting stubgen to run properly. I think the actual code is functional. Thanks for the reminder! |
Add a static fromUri(uri, options?) factory method to the Node.js Operator class, mirroring Rust Operator::from_uri and Python Operator.from_uri (PR #6993). The URI encodes scheme and configuration in a single string, e.g. memory://localhost/ s3://bucket/path?region=us-east-1 Optional extra key-value options can be passed to override or supplement values encoded in the URI. Intentional divergence: none. Direct 1:1 mapping of Rust Operator::from_uri((uri, options)) via IntoOperatorUri. Scope: bindings/nodejs only (lib.rs + generated.d.ts). AI-assisted: od-claude-lead-050121, reviewed by OpenDAL regression team. Co-authored-by: tianyizhuang <tianyizhuang@tianyizhuangs-Mac-mini.local>
Which issue does this PR close?
Closes #6984 .
Rationale for this change
Instantiating Operators from URLs greatly increases flexibility for consumers of the openDAL API as an abstract backend.
What changes are included in this PR?
Add from_uri static methods to the Operator and AsyncOperator classes in the python bindings.
Are there any user-facing changes?
Two new methods are available to users of the python library.
AI Usage Statement
None
Other notes
I am having issues with my development environment which the devcontainer is not making any easier, so I'm flying a bit blind and relying on CI.