Skip to content

Commit

Permalink
doc: improve FileIO doc (#642)
Browse files Browse the repository at this point in the history
Signed-off-by: xxchan <[email protected]>
  • Loading branch information
xxchan authored Sep 23, 2024
1 parent 12e12e2 commit c354983
Show file tree
Hide file tree
Showing 2 changed files with 14 additions and 3 deletions.
16 changes: 13 additions & 3 deletions crates/iceberg/src/io/file_io.rs
Original file line number Diff line number Diff line change
Expand Up @@ -32,16 +32,25 @@ use crate::{Error, ErrorKind, Result};
///
/// All path passed to `FileIO` must be absolute path starting with scheme string used to construct `FileIO`.
/// For example, if you construct `FileIO` with `s3a` scheme, then all path passed to `FileIO` must start with `s3a://`.
///
/// Supported storages:
///
/// | Storage | Feature Flag | Schemes |
/// |--------------------|-------------------|------------|
/// | Local file system | `storage-fs` | `file` |
/// | Memory | `storage-memory` | `memory` |
/// | S3 | `storage-s3` | `s3`, `s3a`|
/// | GCS | `storage-gcs` | `gs` |
#[derive(Clone, Debug)]
pub struct FileIO {
inner: Arc<Storage>,
}

impl FileIO {
/// Try to infer file io scheme from path.
/// Try to infer file io scheme from path. See [`FileIO`] for supported schemes.
///
/// If it's a valid url, for example http://example.org, url scheme will be used.
/// If it's not a valid url, will try to detect if it's a file path.
/// - If it's a valid url, for example `s3://bucket/a`, url scheme will be used, and the rest of the url will be ignored.
/// - If it's not a valid url, will try to detect if it's a file path.
///
/// Otherwise will return parsing error.
pub fn from_path(path: impl AsRef<str>) -> crate::Result<FileIOBuilder> {
Expand Down Expand Up @@ -111,6 +120,7 @@ pub struct FileIOBuilder {

impl FileIOBuilder {
/// Creates a new builder with scheme.
/// See [`FileIO`] for supported schemes.
pub fn new(scheme_str: impl ToString) -> Self {
Self {
scheme_str: Some(scheme_str.to_string()),
Expand Down
1 change: 1 addition & 0 deletions crates/iceberg/src/io/storage.rs
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ impl Storage {
Scheme::Gcs => Ok(Self::Gcs {
config: super::gcs_config_parse(props)?.into(),
}),
// Update doc on [`FileIO`] when adding new schemes.
_ => Err(Error::new(
ErrorKind::FeatureUnsupported,
format!("Constructing file io from scheme: {scheme} not supported now",),
Expand Down

0 comments on commit c354983

Please sign in to comment.