Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove (Orig)Datablocks from Dataset schemas and DTO #1155

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

fpotier
Copy link
Member

@fpotier fpotier commented Apr 8, 2024

Description

#1137

Tests included/Docs Updated?

  • Included for each change/fix?
  • Passing? (Merge will not be approved unless this is checked)
  • Docs updated?
  • New packages used/requires npm install?
  • Toggle added for new features?

@fpotier fpotier changed the title Remove from Dataset schemas and DTO Remove (Orig)Datablocks from Dataset schemas and DTO Apr 8, 2024
@sbliven
Copy link
Contributor

sbliven commented Apr 16, 2024

This should be clearly documented as a breaking change.

@fpotier
Copy link
Member Author

fpotier commented Apr 16, 2024

@sbliven is it really tho? These attributes weren't in the old schema and unless I'm missing something, they're not used

@consolethinks
Copy link

consolethinks commented Apr 26, 2024

class Dataset(Ownable):
    """
    A dataset in SciCat, base class for derived and raw datasets
    """

    pid: Optional[str] = None
    classification: Optional[str] = None
    contactEmail: str
    creationTime: str  # datetime
    datasetName: Optional[str] = None
    description: Optional[str] = None
    history: Optional[
        List[dict]
    ] = None  # list of foreigh key ids to the Messages table
    instrumentId: Optional[str] = None
    isPublished: Optional[bool] = False
    keywords: Optional[List[str]] = None
    license: Optional[str] = None
    numberOfFiles: Optional[int] = None
    numberOfFilesArchived: Optional[int] = None
    orcidOfOwner: Optional[str] = None
    packedSize: Optional[int] = None
    owner: str
    ownerEmail: Optional[str] = None
    sharedWith: Optional[List[str]] = None
    size: Optional[int] = None
    sourceFolder: str
    sourceFolderHost: Optional[str] = None
    techniques: Optional[List[dict]] = None  # with {'pid':pid, 'name': name} as entries
    type: DatasetType
    validationStatus: Optional[str] = None
    version: Optional[str] = None
    scientificMetadata: Optional[Dict] = None


class RawDataset(Dataset):
    """
    Raw datasets from which derived datasets are... derived.
    """

    principalInvestigator: Optional[str] = None
    creationLocation: Optional[str] = None
    type: DatasetType = DatasetType.raw
    dataFormat: Optional[str] = None
    endTime: Optional[str] = None  # datetime
    sampleId: Optional[str] = None
    proposalId: Optional[str] = None


class DerivedDataset(Dataset):
    """
    Derived datasets which have been generated based on one or more raw datasets
    """

    investigator: str
    inputDatasets: List[str]
    usedSoftware: List[str]
    jobParameters: Optional[dict] = None
    jobLogData: Optional[str] = None
    type: DatasetType = DatasetType.derived

not sure about other components, or whether this was at all present in v3, but at least no datablock related attributes were part of the dataset model in PySciCat.

@sbliven
Copy link
Contributor

sbliven commented Apr 29, 2024

Ok, should be fine then

@sbliven
Copy link
Contributor

sbliven commented Sep 10, 2024

@fpotier Can you rebase this PR to the current master?

@fpotier
Copy link
Member Author

fpotier commented Sep 10, 2024

@sbliven I can do it but I'm not sure it's still relevant. I discussed this PR with @nitrosx a few months ago.
If I remember correctly, the idea was to keep the [Orig]Datablocks in the response but remove them from the DTOs used for creation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants