Skip to content

Commit

Permalink
Implemented build process (#57)
Browse files Browse the repository at this point in the history
* Removed vert and frag files from manifest

These can be implemented in code

* Fixed versioneer for the src layout

* Ignore the working directory

* Refactored main and fixed logging

Refactored the main script.
Almost all logging messages should now go via the logging module.
Python stdout and stderr now go to the logging module.
The above two mean that print, warnings, errors and qt messages get logged to the console and saved to the log file with the format specified by the user.

* Refactored setup.py

* Switched to versioneer

versioneer-518 was a stop-gap until versioneer supported the 518 system.

* Refactored logging

basicConfig does everything that I was doing manually

* Refactored main script

The main script is now a bit more defensive.
It tries to log the errors via the logging module.
If that fails it manually prints to the console and saves to a file.

* Refactored

* Removed sys.exit on error

It seems that PyInstaller catches python exceptions and displays its own error dialog.
This will be useful for users who have not extracted the package correctly.

* Enable the console on windows

* Modified the paths

The old default path structure was a bit confusing on windows.
All user files are now stored in subdirectories of one directory which should make finding the data easier.
It should also minimise conflicting file paths.

* Moved the console logic into main

These two functions had overlapping behaviour so it made sense to merge them.
Streamlined the logging setup

* Removed the plugin job thread

This is no longer used

* Added diagnostic output for plugins

* Added readable str and repr for Requirement

* Added build script

* Added missing files to the pyinstaller hook

* Fixed some promise issues

Some promises were failing silently.
This should make it clearer when a promise function crashes.

* Fixed mirrored texture

* Replaced time.sleep with QTimer

time.sleep blocks the thread (in this case the main thread) until it is finished which delays the application opening.
QTimer does not block the main thread.

* Added an option to suppress errors with DisplayException

* Improved error handling

Only suppress errors that subclass Exception.
Log exceptions displayed with DisplayException.
Enclosed all slots in RPC in exception dialogs.
Set a limit of 20 attempts to spawn the broker process.

* Reformatted

* Set up the translator for the broker

This is used in the traceback dialog.

* Don't paint if the context is not correct

If the paint call is run manually the context may be None or a different context which usually leads to an access violation and crash.

* Fixed pip command

* Fixed upload glob

* Hopefully fix fpm issue

* Only remove keys if they have been added

In some cases something happens that means the key is not in the set.
Only remove it if it is present.

* Fix dtype error

* Added selection plugin init

This is required to import the plugin and errors occur without it

* Bumped qt version

* Fixed int size error correctly

The previous fix fixed the issue on mac but broke it on windows.
This changes the various int and long terms which have varying definitions depending on the platform to the fixed width types.
This should fix the issue and make the code clearer to read.

* Refactored invoke to work on any thread

Invoke can now call a function on any Qt thread.
It still defaults to the main thread.
Refactored a little.

* Moved resource pack opengl initialisation to the main thread

The Qt library warns of issues if the context is used across threads.
Moved the opengl initialisation to the main thread via invoke.

* Ensure the RenderResourcePackContainer exists on the main thread

* Refactored level geometry

Creation and deletion of VBOs is now done on the main thread.
Some more work needs to be done to make sure the GPU data is unloaded automatically.

* Exit broker don't crash

If a broker process was spawned but one already existed this would display a popup to the user.
Switched to sys.exit to just exit out of the application.

* Fixed SharedVBOManager on the main thread

If this was called from the main thread during another call it would not restore the original context.
This should now restore the original context once it is finished.

* Fixed some rendering issue on MacOS

For some reason bindAttributeLocation is required on MacOS otherwise the attributes are not assigned correctly.
Use GLSL 150 which is the shader version for OpenGL 3.2 which is the version we are targeting. 130 did not work on MacOS.

* Initialise OpenGL state per model

* Fixed rendering artifacts

Sort chunks from furthest to nearest so that transparent elements are drawn in the correct order

* Reformatted

* Fixed layout warning

* Include all OpenGL modules

On my linux install EGL is required which is not included.

* Partially fixed camera issues

Wayland does not support setPos which makes the camera uncontrollable.
This doesn't fix all the problems in wayland but it does make it usable.

* Made glob path more specific

* Added clearer installer file names

* Try to fix upload paths

* Upload to pypi

Added a section in the build script to upload to pypi
Skip if ubuntu because this fails uploading to pypi

* Added python prefix
  • Loading branch information
gentlegiantJGC authored Aug 29, 2023
1 parent 0c85d96 commit 74f5bab
Show file tree
Hide file tree
Showing 39 changed files with 857 additions and 431 deletions.
66 changes: 66 additions & 0 deletions .github/workflows/python-build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions

name: Build

on:
release:
types: [published]

jobs:
deploy:
runs-on: ${{ matrix.cfg.os }}
defaults:
run:
shell: bash

strategy:
fail-fast: false
matrix:
cfg:
- {os: windows-latest, python-version: '3.10', architecture: x64}
- {os: macos-latest, python-version: '3.10', architecture: x64}
- {os: ubuntu-latest, python-version: '3.10', architecture: x64}

steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.cfg.python-version }} ${{ matrix.cfg.architecture }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.cfg.python-version }}
architecture: ${{ matrix.cfg.architecture }}

- name: Install dependencies
run: |
pip install --upgrade pip
pip install setuptools wheel twine pyinstaller~=5.6 build
- name: Build and publish to PyPi
if: matrix.cfg.os != 'ubuntu-latest'
env:
TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
TWINE_PASSWORD: ${{ secrets.AMULET_EDITOR_PYPI_PASSWORD }}
run: |
python -m build -C--build-option=--freeze-first=True
python -m twine upload dist/amulet[_-]editor* --skip-existing
- name: Install Ubuntu dependencies
if: matrix.cfg.os == 'ubuntu-latest'
run: |
sudo apt-get install ruby-dev build-essential
sudo gem i fpm -f
- name: Create Installer
run: |
pip install git+https://github.com/Amulet-Team/python-build-tool.git
pip install --upgrade --upgrade-strategy eager -e .
pbt freeze
pbt installer
- name: Upload Release Asset
uses: softprops/action-gh-release@v1
with:
files: |
target/*.exe
target/*.deb
target/*.dmg
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -125,3 +125,5 @@ venv.bak/
*.ui.autosave

*.ini

working/
2 changes: 0 additions & 2 deletions MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,5 @@
recursive-include src/amulet_editor *.py *.pyi *.pyx *.pxd
# Images
recursive-include src/amulet_editor *.jpg *.png *.svg *.ico
# Opengl
recursive-include src/amulet_editor *.vert *.frag
# Misc
recursive-include src/amulet_editor *.mcmeta *.lang *.json *.qss
3 changes: 0 additions & 3 deletions build_system/build/settings/linux.json

This file was deleted.

5 changes: 3 additions & 2 deletions build_system/build/settings/mac.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
{
"mac_bundle_identifier": "",
"show_console_window": false
"installer": "${app_name}-${version}-macos-x64.dmg",
"show_console_window": false,
"mac_bundle_identifier": "com.amuletmc.amulet-editor"
}
3 changes: 3 additions & 0 deletions build_system/build/settings/ubuntu.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"installer": "${app_name}-${version}-ubuntu.deb"
}
4 changes: 2 additions & 2 deletions build_system/build/settings/windows.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{
"installer": "${app_name}-${version}-Installer.exe",
"show_console_window": false
"installer": "${app_name}-${version}-windows-x64.exe",
"show_console_window": true
}
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ requires = [
"setuptools >= 42",
"wheel",
"cython >= 3.0.0a9",
"versioneer-518",
"versioneer",
"numpy ~= 1.17"
]
build-backend = "setuptools.build_meta"
2 changes: 1 addition & 1 deletion setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ gui_scripts =
[versioneer]
VCS = git
style = pep440
versionfile_source = amulet_editor/_version.py
versionfile_source = src/amulet_editor/_version.py
versionfile_build = amulet_editor/_version.py
tag_prefix =
parentdir_prefix = amulet_editor-
29 changes: 17 additions & 12 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,18 +12,21 @@


first_party = {
"amulet-core",
"amulet-nbt",
"amulet_core",
"amulet_nbt",
"pymctranslate",
"minecraft-resource-pack",
"minecraft_resource_pack",
}


def freeze_requirements(packages: List[str]) -> List[str]:
# Pip install the requirements to find the newest compatible versions
# This makes sure that the source versions are using the same dependencies as the compiled version.
# This also makes sure that the source version is using the newest version of the dependency.
if any("~=" in r and r.split("~=", 1)[0].lower() in first_party for r in packages):
if any(
"~=" in r and r.split("~=", 1)[0].lower().replace("-", "_") in first_party
for r in packages
):
print("pip-install")
try:
# make sure pip is up to date
Expand All @@ -40,12 +43,14 @@ def freeze_requirements(packages: List[str]) -> List[str]:
.strip()
.split("\n")
)
requirements_map = {r.split("==")[0].lower(): r for r in installed}
requirements_map = {
r.split("==")[0].lower().replace("-", "_"): r for r in installed
}

print(installed, requirements_map)
for index, requirement in enumerate(packages):
if "~=" in requirement:
lib = requirement.split("~=")[0].strip().lower()
lib = requirement.split("~=")[0].strip().lower().replace("-", "_")
if lib in first_party and lib in requirements_map:
packages[index] = requirements_map[lib]
print(f"Modified packages to {packages}")
Expand Down Expand Up @@ -122,29 +127,29 @@ def get_openmp_args_for(arg) -> Tuple[List[str], List[str]]:
# so that it doesn't error. It doesn't actually use it.
class SDist(cmdclass["sdist"]):
user_options = cmdclass["sdist"].user_options + [
("find-libs=", None, ""),
("freeze-first=", None, ""),
]

def initialize_options(self):
super().initialize_options()
self.find_libs = None
self.freeze_first = None


class BDistWheel(bdist_wheel):
user_options = bdist_wheel.user_options + [
(
"find-libs=",
"freeze-first=",
None,
"Find and fix the newest version of first party libraries. Only used internally.",
"Find and freeze the newest version of first party libraries. Only used internally.",
),
]

def initialize_options(self):
super().initialize_options()
self.find_libs = None
self.freeze_first = None

def finalize_options(self):
if self.find_libs:
if self.freeze_first:
self.distribution.install_requires = freeze_requirements(
list(self.distribution.install_requires)
)
Expand Down
Loading

0 comments on commit 74f5bab

Please sign in to comment.