Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Typeshare support for Python #169

Open
wants to merge 60 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 54 commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
90f7537
Add experimental typeshare support for Python
hculea Apr 30, 2024
ee5673c
Resolve clippy errors
hculea May 28, 2024
78bca98
Resolve merge conflicts with latest main
hculea May 28, 2024
dfd6522
Resolve merge conflicts with latest main
hculea May 28, 2024
0c14914
Add python as an option for the --lang flag
hculea May 28, 2024
439783b
Add support for todo and priorly unsupported types
hculea Jun 10, 2024
d7f12d0
Leverage mutable self to remove RefCell around Module
hculea Jun 10, 2024
2039818
Fix ParsedRusthThing typo
hculea Jun 10, 2024
48a8adb
Merge branch 'main' into hculea/add-python-support-in-typeshare
MOmarMiraj Jul 23, 2024
1427223
Merge branch '1Password:main' into hculea/add-python-support-in-types…
AndyTitu Aug 5, 2024
c9f4be4
Use serialization alias in renaming fields in Python
hculea Aug 5, 2024
ad6645a
Merge branch '1Password:main' into hculea/add-python-support-in-types…
hculea Aug 20, 2024
459f596
Fix python serializing and deserialzing using pydantic Field aliasing…
AndyTitu Aug 20, 2024
fde8907
Fix indentation of writing config dict
AndyTitu Aug 20, 2024
ead7d3a
added None default to Optional python type
amandayu1 Aug 20, 2024
a74e4ce
Updated None default condition
amandayu1 Aug 22, 2024
9b5c4be
fixing optional field set to default
amandayu1 Aug 22, 2024
628608d
add fix for missing fields
amandayu1 Aug 22, 2024
8519dea
added fix
amandayu1 Aug 22, 2024
6786aad
changed fix
amandayu1 Aug 22, 2024
2d0e627
testing new approach
amandayu1 Aug 22, 2024
169b380
Merge pull request #1 from hculea/optional-fields-default-to-none
amandayu1 Aug 22, 2024
e6bea87
added clarifying comment for None default
amandayu1 Aug 22, 2024
d51d903
set optional field to None by default
amandayu1 Aug 27, 2024
aa2213d
Bring Python fork up to date with main
hculea Oct 1, 2024
eeaa0d1
Run cargo fmt
hculea Oct 1, 2024
e79b7b6
fix duplicated " = None = None"
CheatCod Oct 1, 2024
d98a3e2
fix duplicate
CheatCod Oct 1, 2024
677cf13
add tests
CheatCod Oct 1, 2024
df4a611
add comments to test
CheatCod Oct 1, 2024
77b7239
minor formatting
CheatCod Oct 1, 2024
b3536c8
Remove level of indentation of type alias on comments
MOmarMiraj Oct 2, 2024
45e9e5f
Refactor Python Codegen to correctly top sort file and use typeshare …
MOmarMiraj Oct 3, 2024
c8026b0
Add Feature Flag for Python TS (#3)
MOmarMiraj Oct 28, 2024
e419246
Draft: Change tagged enum generation in python (#2)
CheatCod Oct 31, 2024
53b25db
Merge remote-tracking branch 'upstream/main' into hculea/add-python-s…
MOmarMiraj Oct 31, 2024
fb666ca
Add python support and update cargo lock
MOmarMiraj Oct 31, 2024
40206a7
fmt fix and fix rust doc
MOmarMiraj Oct 31, 2024
7184b52
update test cases
MOmarMiraj Oct 31, 2024
3d447df
Add mention of Python support in README
hculea Nov 4, 2024
a251db1
remove trailing comma
MOmarMiraj Nov 7, 2024
03d600b
Merge branch 'hculea/add-python-support-in-typeshare' of github.com:h…
MOmarMiraj Nov 7, 2024
61b6afd
Update unit enum to use enum values
MOmarMiraj Nov 7, 2024
9810e7b
fmt fix
MOmarMiraj Nov 7, 2024
92f6922
Remove config dict and allow unit enums to be serializble
MOmarMiraj Nov 7, 2024
0f89783
fix tuple variant generation
CheatCod Nov 8, 2024
90793c7
dedup union
CheatCod Nov 8, 2024
40803c4
better error handling
CheatCod Nov 8, 2024
23ecf7f
Merge pull request #4 from hculea/peter/fix-tuple-variant-generation
CheatCod Nov 8, 2024
ce97007
remove once-cell; use index-set
CheatCod Nov 12, 2024
260fa75
update tests
CheatCod Nov 12, 2024
d4fabb6
add comments on unused function for generics
CheatCod Nov 12, 2024
83001f7
refactor how union works. Note this requires the struct to be externa…
CheatCod Nov 14, 2024
6afd25a
proper rename
CheatCod Nov 14, 2024
0c9b8ac
use literal for tag keys
CheatCod Nov 15, 2024
a10d3e7
generate comments properly
CheatCod Nov 15, 2024
3df3ece
fix type aliasing issue
CheatCod Nov 15, 2024
533c75f
remove unneeded newlines
CheatCod Nov 15, 2024
4a0bb0e
fix to confrom python's wonderful doc conventions
CheatCod Nov 15, 2024
a49914a
use enum for type
CheatCod Nov 15, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
418 changes: 182 additions & 236 deletions Cargo.lock

Large diffs are not rendered by default.

5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ _And in the darkness, compile them_ 💍


Do you like manually managing types that need to be passed through an FFI layer, so that your code doesn't archaically break at runtime? Be honest, nobody does. Typeshare is here to take that burden away from you! Leveraging the power of the `serde` library, Typeshare is a tool that converts your
Rust types into their equivalent forms in Swift, Go**, Kotlin, Scala and Typescript, keeping
Rust types into their equivalent forms in Swift, Go**, Python**, Kotlin, Scala and Typescript, keeping
your cross-language codebase in sync. With automatic implementation for serialization and deserialization on both sides of the FFI, Typeshare does all the heavy lifting for you. It can even handle generics and convert effortlessly between standard libraries in different languages!

**A few caveats. See [here](#a-quick-refresher-on-supported-languages) for more details.
Expand Down Expand Up @@ -98,12 +98,13 @@ Are you getting weird deserialization issues? Did our procedural macro throw a c
- Swift
- Typescript
- Go**
- Python**

If there is a language that you want Typeshare to generate definitions for, you can either:
1. Open an issue in this repository requesting your language of choice.
2. Implement support for that language and open a PR with your implementation. We would be eternally grateful! 🙏

** Right now, Go support is experimental. Enable the `go` feature when installing typeshare-cli if you want to use it.
** Right now, Go and Python support is experimental. Enable the `go` or `python` features, respectively, when installing typeshare-cli if you want to use these.

## Credits

Expand Down
1 change: 1 addition & 0 deletions cli/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ path = "src/main.rs"

[features]
go = []
python = []

[dependencies]
clap = { version = "4.5", features = [
Expand Down
6 changes: 5 additions & 1 deletion cli/data/tests/mappings_config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,8 @@
"DateTime" = "String"

[go.type_mappings]
"DateTime" = "string"
"DateTime" = "string"

[python.type_mappings]
"DateTime" = "datetime"
"Url" = "AnyUrl"
2 changes: 2 additions & 0 deletions cli/src/args.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ pub enum AvailableLanguage {
Typescript,
#[cfg(feature = "go")]
Go,
#[cfg(feature = "python")]
Python,
}

#[derive(clap::Parser)]
Expand Down
14 changes: 14 additions & 0 deletions cli/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,13 @@ use std::{

const DEFAULT_CONFIG_FILE_NAME: &str = "typeshare.toml";

#[derive(Default, Serialize, Deserialize, PartialEq, Eq, Debug)]
#[serde(default)]
#[cfg(feature = "python")]
pub struct PythonParams {
pub type_mappings: HashMap<String, String>,
}

#[derive(Default, Serialize, Deserialize, Debug, PartialEq, Eq)]
#[serde(default)]
pub struct KotlinParams {
Expand Down Expand Up @@ -64,6 +71,8 @@ pub(crate) struct Config {
pub typescript: TypeScriptParams,
pub kotlin: KotlinParams,
pub scala: ScalaParams,
#[cfg(feature = "python")]
pub python: PythonParams,
#[cfg(feature = "go")]
pub go: GoParams,
#[serde(skip)]
Expand Down Expand Up @@ -160,6 +169,11 @@ mod test {
assert_eq!(config.kotlin.type_mappings["DateTime"], "String");
assert_eq!(config.scala.type_mappings["DateTime"], "String");
assert_eq!(config.typescript.type_mappings["DateTime"], "string");
#[cfg(feature = "python")]
{
assert_eq!(config.python.type_mappings["Url"], "AnyUrl");
assert_eq!(config.python.type_mappings["DateTime"], "datetime");
}
#[cfg(feature = "go")]
assert_eq!(config.go.type_mappings["DateTime"], "string");
}
Expand Down
19 changes: 15 additions & 4 deletions cli/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,13 @@ use clap_complete::Generator;
use ignore::{overrides::OverrideBuilder, types::TypesBuilder, WalkBuilder};
use log::error;
use rayon::iter::ParallelBridge;
use typeshare_core::language::GenericConstraints;
#[cfg(feature = "go")]
use typeshare_core::language::Go;
#[cfg(feature = "python")]
use typeshare_core::language::Python;
use typeshare_core::{
language::{
CrateName, GenericConstraints, Kotlin, Language, Scala, SupportedLanguage, Swift,
TypeScript,
},
language::{CrateName, Kotlin, Language, Scala, SupportedLanguage, Swift, TypeScript},
parser::ParsedData,
};

Expand Down Expand Up @@ -78,6 +78,8 @@ fn main() -> anyhow::Result<()> {
args::AvailableLanguage::Typescript => SupportedLanguage::TypeScript,
#[cfg(feature = "go")]
args::AvailableLanguage::Go => SupportedLanguage::Go,
#[cfg(feature = "python")]
args::AvailableLanguage::Python => SupportedLanguage::Python,
},
};

Expand Down Expand Up @@ -210,6 +212,15 @@ fn language(
SupportedLanguage::Go => {
panic!("go support is currently experimental and must be enabled as a feature flag for typeshare-cli")
}
#[cfg(feature = "python")]
SupportedLanguage::Python => Box::new(Python {
type_mappings: config.python.type_mappings,
..Default::default()
}),
#[cfg(not(feature = "python"))]
SupportedLanguage::Python => {
panic!("python support is currently experimental and must be enabled as a feature flag for typeshare-cli")
}
}
}

Expand Down
1 change: 1 addition & 0 deletions cli/src/parse.rs
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ fn output_file_name(language_type: SupportedLanguage, crate_name: &CrateName) ->
SupportedLanguage::Scala => snake_case(),
SupportedLanguage::Swift => pascal_case(),
SupportedLanguage::TypeScript => snake_case(),
SupportedLanguage::Python => snake_case(),
}
}

Expand Down
2 changes: 2 additions & 0 deletions core/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ thiserror = "1"
itertools = "0.12"
lazy_format = "2"
joinery = "2"
topological-sort = { version = "0.2.2"}
convert_case = { version = "0.6.0"}
log.workspace = true
flexi_logger.workspace = true

Expand Down
57 changes: 57 additions & 0 deletions core/data/tests/anonymous_struct_with_rename/output.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
"""
Generated by typeshare 1.12.0
"""
from __future__ import annotations

from enum import Enum
from pydantic import BaseModel, ConfigDict, Field
from typing import List, Union


class AnonymousStructWithRenameListInner(BaseModel):
"""
Generated type representing the anonymous struct variant `List` of the `AnonymousStructWithRename` Rust enum
"""
list: List[str]


class AnonymousStructWithRenameLongFieldNamesInner(BaseModel):
"""
Generated type representing the anonymous struct variant `LongFieldNames` of the `AnonymousStructWithRename` Rust enum
"""
model_config = ConfigDict(populate_by_name=True)

some_long_field_name: str
and_: bool = Field(alias="and")
but_one_more: List[str]


class AnonymousStructWithRenameKebabCaseInner(BaseModel):
"""
Generated type representing the anonymous struct variant `KebabCase` of the `AnonymousStructWithRename` Rust enum
"""
model_config = ConfigDict(populate_by_name=True)

another_list: List[str] = Field(alias="another-list")
camel_case_string_field: str = Field(alias="camelCaseStringField")
something_else: bool = Field(alias="something-else")


class AnonymousStructWithRenameTypes(str, Enum):
LIST = "list"
LONG_FIELD_NAMES = "longFieldNames"
KEBAB_CASE = "kebabCase"

class AnonymousStructWithRenameList(BaseModel):
type: AnonymousStructWithRenameTypes = AnonymousStructWithRenameTypes.LIST
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's possible I'm misunderstanding pydantic, but shouldn't this be a Literal parameter? I had understood this = to just set a default value.

https://docs.pydantic.dev/2.0/usage/types/unions/

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

content: AnonymousStructWithRenameListInner

class AnonymousStructWithRenameLongFieldNames(BaseModel):
type: AnonymousStructWithRenameTypes = AnonymousStructWithRenameTypes.LONG_FIELD_NAMES
content: AnonymousStructWithRenameLongFieldNamesInner

class AnonymousStructWithRenameKebabCase(BaseModel):
type: AnonymousStructWithRenameTypes = AnonymousStructWithRenameTypes.KEBAB_CASE
content: AnonymousStructWithRenameKebabCaseInner

AnonymousStructWithRename = Union[AnonymousStructWithRenameList, AnonymousStructWithRenameLongFieldNames, AnonymousStructWithRenameKebabCase]
47 changes: 47 additions & 0 deletions core/data/tests/can_apply_prefix_correctly/output.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
"""
Generated by typeshare 1.12.0
"""
from __future__ import annotations

from enum import Enum
from pydantic import BaseModel
from typing import Dict, List, Union


class ItemDetailsFieldValue(BaseModel):
hello: str


class AdvancedColorsTypes(str, Enum):
STRING = "String"
NUMBER = "Number"
NUMBER_ARRAY = "NumberArray"
REALLY_COOL_TYPE = "ReallyCoolType"
ARRAY_REALLY_COOL_TYPE = "ArrayReallyCoolType"
DICTIONARY_REALLY_COOL_TYPE = "DictionaryReallyCoolType"

class AdvancedColorsString(BaseModel):
t: AdvancedColorsTypes = AdvancedColorsTypes.STRING
c: str

class AdvancedColorsNumber(BaseModel):
t: AdvancedColorsTypes = AdvancedColorsTypes.NUMBER
c: int

class AdvancedColorsNumberArray(BaseModel):
t: AdvancedColorsTypes = AdvancedColorsTypes.NUMBER_ARRAY
c: List[int]

class AdvancedColorsReallyCoolType(BaseModel):
t: AdvancedColorsTypes = AdvancedColorsTypes.REALLY_COOL_TYPE
c: ItemDetailsFieldValue

class AdvancedColorsArrayReallyCoolType(BaseModel):
t: AdvancedColorsTypes = AdvancedColorsTypes.ARRAY_REALLY_COOL_TYPE
c: List[ItemDetailsFieldValue]

class AdvancedColorsDictionaryReallyCoolType(BaseModel):
t: AdvancedColorsTypes = AdvancedColorsTypes.DICTIONARY_REALLY_COOL_TYPE
c: Dict[str, ItemDetailsFieldValue]

AdvancedColors = Union[AdvancedColorsString, AdvancedColorsNumber, AdvancedColorsNumberArray, AdvancedColorsReallyCoolType, AdvancedColorsArrayReallyCoolType, AdvancedColorsDictionaryReallyCoolType]
67 changes: 67 additions & 0 deletions core/data/tests/can_generate_algebraic_enum/output.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
"""
Generated by typeshare 1.12.0
"""
from __future__ import annotations

from enum import Enum
from pydantic import BaseModel
from typing import List, Union


class ItemDetailsFieldValue(BaseModel):
"""
Struct comment
"""
pass

class AdvancedColorsTypes(str, Enum):
STRING = "String"
NUMBER = "Number"
UNSIGNED_NUMBER = "UnsignedNumber"
NUMBER_ARRAY = "NumberArray"
REALLY_COOL_TYPE = "ReallyCoolType"

class AdvancedColorsString(BaseModel):
type: AdvancedColorsTypes = AdvancedColorsTypes.STRING
content: str

class AdvancedColorsNumber(BaseModel):
type: AdvancedColorsTypes = AdvancedColorsTypes.NUMBER
content: int

class AdvancedColorsUnsignedNumber(BaseModel):
type: AdvancedColorsTypes = AdvancedColorsTypes.UNSIGNED_NUMBER
content: int

class AdvancedColorsNumberArray(BaseModel):
type: AdvancedColorsTypes = AdvancedColorsTypes.NUMBER_ARRAY
content: List[int]

class AdvancedColorsReallyCoolType(BaseModel):
type: AdvancedColorsTypes = AdvancedColorsTypes.REALLY_COOL_TYPE
content: ItemDetailsFieldValue

AdvancedColors = Union[AdvancedColorsString, AdvancedColorsNumber, AdvancedColorsUnsignedNumber, AdvancedColorsNumberArray, AdvancedColorsReallyCoolType]
class AdvancedColors2Types(str, Enum):
STRING = "string"
NUMBER = "number"
NUMBER_ARRAY = "number-array"
REALLY_COOL_TYPE = "really-cool-type"

class AdvancedColors2String(BaseModel):
type: AdvancedColors2Types = AdvancedColors2Types.STRING
content: str

class AdvancedColors2Number(BaseModel):
type: AdvancedColors2Types = AdvancedColors2Types.NUMBER
content: int

class AdvancedColors2NumberArray(BaseModel):
type: AdvancedColors2Types = AdvancedColors2Types.NUMBER_ARRAY
content: List[int]

class AdvancedColors2ReallyCoolType(BaseModel):
type: AdvancedColors2Types = AdvancedColors2Types.REALLY_COOL_TYPE
content: ItemDetailsFieldValue

AdvancedColors2 = Union[AdvancedColors2String, AdvancedColors2Number, AdvancedColors2NumberArray, AdvancedColors2ReallyCoolType]
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
"""
Generated by typeshare 1.12.0
"""
from __future__ import annotations

from enum import Enum
from pydantic import BaseModel
from typing import Union


class SomeEnumTypes(str, Enum):
A = "A"
C = "C"

class SomeEnumA(BaseModel):
type: SomeEnumTypes = SomeEnumTypes.A

class SomeEnumC(BaseModel):
type: SomeEnumTypes = SomeEnumTypes.C
content: int

SomeEnum = Union[SomeEnumA, SomeEnumC]
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
"""
Generated by typeshare 1.12.0
"""
from __future__ import annotations

from enum import Enum
from pydantic import BaseModel
from typing import Union


class AutofilledByUsInner(BaseModel):
"""
Generated type representing the anonymous struct variant `Us` of the `AutofilledBy` Rust enum
"""
uuid: str
"""
The UUID for the fill
"""


class AutofilledBySomethingElseInner(BaseModel):
"""
Generated type representing the anonymous struct variant `SomethingElse` of the `AutofilledBy` Rust enum
"""
uuid: str
"""
The UUID for the fill
"""


class AutofilledByTypes(str, Enum):
US = "Us"
SOMETHING_ELSE = "SomethingElse"

class AutofilledByUs(BaseModel):
type: AutofilledByTypes = AutofilledByTypes.US
content: AutofilledByUsInner

class AutofilledBySomethingElse(BaseModel):
type: AutofilledByTypes = AutofilledByTypes.SOMETHING_ELSE
content: AutofilledBySomethingElseInner

AutofilledBy = Union[AutofilledByUs, AutofilledBySomethingElse]
Loading