Consider adding a image resource container to image load management

[evaristocuesta]

Are you considering to use an image resource container to manage the image load following the flyweight pattern?

I developed a first aproach in the following branch where you can see the idea. Following the flyweight pattern allow you save memory when you use the same image in several game objects.

If you think that this can be useful for the project, I can create a Pull Request to one of your branches.

[bijington]

@evaristocuesta Thank you for this! I really like the idea of sharing resources in this way as it should make a big difference especially in areas where the same resource is used multiple times.

I haven't looked over the branch fully but the general concept looks good! The only bit I am not entirely sure on is the Initialize method. This isn't a criticism I just haven't worked out what lifecycle I want to publicly expose yet. I suspect there will be something like Initialize or an OnAdded method

[evaristocuesta]

@bijington Thank you for considering my proposal.

I was thinking about the lifecycle and I think that it could be a good idea to add LoadResources and Initialized methods to IGameScene. LoadResources and Initialized methods will be called from GameSceneManager.LoadScene

LoadResources: Load the images resources, sound resources and so on which belong to the scene. This function should load the resources asynchronously and call to LoadProgressChanged event to allow show a load progress bar.

Initialized: Add game objects belonging to the scene instead of adding from the constructor of the class. The GameObjectContainer.Add function will call to GameObject.Initialize.

namespace Orbit.Engine;

/// <summary>
/// Interface definition representing a scene or level in a game.
/// </summary>
public interface IGameScene : IDrawable, IGameObjectContainer
{
    /// <summary>
    /// Event raised when load progress changes.
    /// </summary>
    event EventHandler<LoadProgressChangedEventArgs> LoadProgressChanged;

    /// <summary>
    /// Event raised when LoadResources finishes.
    /// </summary>
    event EventHandler<ResourcesLoadedEventArgs> ResourcesLoaded;

    /// <summary>
    /// Event raised when Initialize finishes.
    /// </summary>
    event EventHandler<InitializedEventArgs> Initialized;

    /// <summary>
    /// Load images resources, sound resources and so on which belong to the scene
    /// This function should load the resources asynchronously and call to LoadProgressChanged event 
    /// to allow show a load progress bar
    /// </summary>
    void LoadResources();
    
    /// <summary>
    /// Add game objects belonging to the scene instead of adding from the constructor of the class
    /// The GameObjectContainer.Add function will call to GameObject.Initialize
    /// </summary>
    void Initialize();
}

[bijington]

I like where this is going! I usually try to avoid events however I hadn't considered how to show something like a loading screen let alone actual progress of the load.

So I do like the idea of loading all resources up front, I am just trying to think about how this might be defined. It would be ideal if the game objects themselves still know about the resources they need and not the scene directly, of course some objects won't be directly added to a scene immediately so I am not sure what should happen with their resources (for example in the Orbit game an Asteroid only gets added when it spawns for the first time - perhaps it becomes the job of the spawner to handle the loading of the resource there.)

I would also like to explore whether the initial ramblings in #13 could be incorporated into this lifecycle discussion and perhaps that might ultimately be where the change gets made. I was expecting that objects may need to unload resources too in case some might be heavy in terms of footprint, etc.

Sorry I haven't helped conclude anything yet but I am certainly enjoying the discussion around this

[bijington]

In actual fact I am just playing around with audio support so that is another resource that would be nice to share or even tidy up once the object is removed from a scene.

[evaristocuesta]

It would be ideal if the game objects themselves still know about the resources they need and not the scene directly, of course some objects won't be directly added to a scene immediately so I am not sure what should happen with their resources (for example in the Orbit game an Asteroid only gets added when it spawns for the first time - perhaps it becomes the job of the spawner to handle the loading of the resource there.)

For performance purpose, I think is better to have the resource loaded instead of load it when the a game object is created as an Asteoroid do, because you could notice a lag playing the game when the resource loads the image.

And I see your point of view about game objects know the resoures they need. It is still possible to show a loading screen in this case. But think about develop a resources package tool to package all the resources and load this package in the game, instead of have separate resources. This resource package would be loaded in the game scene and would be availble when the game objects need them.

In actual fact I am just playing around with audio support so that is another resource that would be nice to share or even tidy up once the object is removed from a scene.

Yes, of course. The same idea works for sounds and even sprite sheets for animations.

[bijington]

All of what you have said makes sense! I agree the resources should be loaded up when the scene loads or at least as many as pragmatically possible. I guess I was trying to explore how an Asteroid which has a specific image to render registers this with the scene so when it gets loaded the dependent image will be loaded. I think your approach will likely cover it or if not it shouldn't take much to cater for that scenario 👍

I'm away for a week now but I'll think on this topic. I'm short the proposal you came up with did look good. When I get back I'll try and see if I can wrap my head around the scenario above and also my brief investigation into other resources such as audio.

On the topic of audio do you think having a general purpose resource container is best or perhaps some specific ones for each type of resource?

[evaristocuesta]

On the topic of audio do you think having a general purpose resource container is best or perhaps some specific ones for each type of resource?

I think that it could be a general purpose resource container. Something like the following code:

public interface IResourceContainer
{
    void AddImage(string key, string path);

    IImageResource Get(string key);

    void RemoveImage(string key);

    void ClearImages();

    void AddSpriteSheet(string key, string path);

    ISpriteSheetResource Get(string key);

    void RemoveSpriteSheet(string key);

    void ClearSpriteSheets();

    void AddSound(string key, string path);

    ISoundResource Get(string key);

    void RemoveSound(string key);

    void ClearSounds();

    void AddVideo(string key, string path);

    IVideoResource Get(string key);

    void RemoveVideo(string key);

    void ClearVideos();

}

But it would work with a specific container for each type as well.

Anyway, I would avoid casting of different types of resources.

[bijington]

@evaristocuesta apologies for being quiet on this thread for some time. I had to take a much needed break from the side project work and slowly build up the time I could spend on them again. I do hope to get back into this project in the coming weeks.

I like this general concept and would certainly like to proceed with implementing the flyweight pattern. Are you still available to potentially assist with implementing it? I'd be happy to stick with supporting images initially and then see where to build on from there as audio and video will likely require additional dependencies and I would like to keep the framework as lightweight as possible for now.

[evaristocuesta]

@bijington even though I have less time than last summer, I'm still available to assist with implementing this idea.

[bijington]

@evaristocuesta that is awesome! Thank you so much!

[bijington]

I'd be happy to kickstart it and then maybe see what you think to the implementation if it helps?

[evaristocuesta]

I developed a first aproach in August. You can see it in this branch. If you want, we can start from this branch.

[bijington]

That certainly looks like a decent starting point if it isn't the full solution!

[redogit]

I developed this approach as extension methods to GameObject. it accomplishes grouping and loading at the same time. Can load mp3 or png out of box.

/// Resource loading is designed to store resources in groups
/// it discards file name and groups on folder location.
/// For instance:
/// Folder/Walking/Left/*  and Folder/Walking/Right/* would be grouped separately
/// With the Key(s) Folder/Walking/Left & Folder/Walking/Right Respectively.
public static class GameObjectExtensions
{
    public delegate Output ResourceHandlerFunc<out Output>(Stream input);

    private const string Images = "Images";
    private const string ImgExt = ".png";
    private const string Audio = "Audio";
    private const string AudExt = ".mp3";

    private static readonly ResourceHandlerFunc<IImage> iimageHandlerFunc = (stream) =>
    {
#if WINDOWS
        return new W2DImageLoadingService().FromStream(stream);
#else
        return PlatformImage.FromStream(stream);
#endif
    };

    private static readonly ResourceHandlerFunc<IAudioPlayer> audioHandlerFunc = (stream) => AudioManager.Current.CreatePlayer(stream);

    public static ILookup<string, Lazy<IImage>> LoadImages(this GameObject gameObject)
    {
        return ProcessResourceStreamsWith(AcquireResourceStreams(gameObject, Images, ImgExt), iimageHandlerFunc);
    }
    public static ILookup<string, Lazy<IAudioPlayer>> LoadAudio(this GameObject gameObject)
    {
        return ProcessResourceStreamsWith(AcquireResourceStreams(gameObject, Audio, AudExt), audioHandlerFunc);
    }
    public static ILookup<string, Lazy<T>> DynamicResourceLoader<T>(this GameObject gameObject, string resourceFolder, string resourceExtension, ResourceHandlerFunc<T> HandlerFunction)
    {
        return ProcessResourceStreamsWith(AcquireResourceStreams(gameObject, resourceFolder, resourceExtension), HandlerFunction);
    }
    private static Dictionary<string, Stream> AcquireResourceStreams(this GameObject @object, string resourceFolder, string resourceExtension)
    {
        Dictionary<string, Stream> resources = new();
        var assembly = @object.GetType().GetTypeInfo().Assembly;
        foreach (var resourcesFor in assembly
            .GetManifestResourceNames()
            .Where(x => x.Contains(resourceFolder) && x.Contains(@object.GetType().Name) && x.Contains(resourceExtension)))
        {
            resources.Add(resourcesFor, assembly.GetManifestResourceStream(resourcesFor));
        }
        return resources;
    }
    private static ILookup<string, Lazy<T>> ProcessResourceStreamsWith<T>(in Dictionary<string, Stream> resources, ResourceHandlerFunc<T> func)
    {
    return (from key in resources
            let value = key.Value
            // remove extension
            let firstStage = key.Key[..key.Key.LastIndexOf('.')]
            // remove file name
            let secondStage = firstStage[..firstStage.LastIndexOf('.')]
            select (secondStage, new Lazy<T>(func(value)))).ToLookup(x => x.secondStage, x => x.Item2);        
    }
}