Scope of this repository

[kinnay]

Currently, this repository implements everything that is required to access NEX services from a PC. Should it support other services as well? I've been thinking about this for a while.

First, some observations:

• It's hardly possible to use NEX without talking to DAuth, AAuth and BaaS, so we have to implement those services anyway.
• Some people do want to expand the scope of this repository (for example #85 and #87).

Also:

• The wiki documents much more services than NEX: Pia, NPLN, AC:NH, just to name a few.
• I've been working on LDN and NPLN in different repositories. I've also tried to implement Pia in this repository a few times, but was never able to get a fully working implementation because of its complexity.

I'm not against adding support for other services, but I also want to avoid feature creep. And:

• The NEX implementation works quite well. I'm hesitant to implement something like Pia here, because its initial implementation will inevitably have problems. If it's in a different repository, I don't care. It can become more mature at its own pace. I just don't want to provide well-functioning services and half-broken services in the same repository.
• Some services have a completely different design. It doesn't make sense to implement LDN or provide NPLN protobuf files in this repository.

I've been thinking about creating a github organization. This would have the following benefits:

• All related repositories would be in one place, without unrelated repositories (example) filling the repository list.
• As long as this repository is under my personal account, I want to be in control over the code that is committed. When it is moved to an organization, I might be less hesitant to add collaborators.

As far as I know, github automatically redirects the old repository when it is transferred to an organization, so existing links will remain intact.

If we do this, a few questions remain:

• What would be the scope of the organization? Of course it would provide client code + documentation, but it might make sense to provide some server code as well?
• How do we organize the code? Pia should probably get its own repository. Should we split the HTTP services into their own repository as well? They are independent from NEX after all. And what about generic code such as miis.py?
• What would be a good name for the organization?

Just wanted to dump my thoughts here. If you want to share your opinion, leave a comment below.

[jonbarrow]

If you go the organization route you could follow a similar repository structure that we use at Pretendo, though we entirely focus on the server side of things not the client side

We have many repositories all dedicated to specific functions. For example we have the following for NEX servers:

• https://github.com/PretendoNetwork/nex-go is our base NEX server implementation. It only handles raw packet handling and emitting events based on the packet type it receives. I've even been thinking about breaking this apart even further and having a dedicated PRUDP repository that handles the PRUDP side of things, leaving nex-go to be the server implementation
• https://github.com/PretendoNetwork/nex-protocols-go is our implementation of each NEX protocol. All this repository does is provide an interface for use to register protocols on a given server and provide the library handler functions to be called for different protocol methods
• https://github.com/PretendoNetwork/nex-protocols-common-go is where we have standardized implementations of common NEX protocols which are used on all/many of the servers such as the authentication and secure connection protocols. This has allowed us to write fully functional authentication servers in as little as 40 lines of code

Then on top of that we have dedicated repositories for each server such as the account server, the BOSS server, etc

[jonbarrow]

Some people do want to expand the scope of this repository (for example #85 and #87)

To be clear (since my issue was mentioned here), my goal is not to expand the scope of the repository. The way it looks, the wiki is supposed to be full documentation on all things Nintendo related? I only wanted to complete this documentation

On that note though, I think if we do go with an org, it would be important to make the wiki into maybe it's own dedicated repository. That way other people can directly contribute, rather than making issues and hoping things get updated

For example #57 is an on-going list of wiki issues which we have no way of fixing ourselves

My team over at Pretendo quite often makes new discoveries or finds new issues which should be documented, but doing so is difficult right now without having fragmented documentation. Right now we keep notes on Discord while making issues here and DMing you on Discord, but that is inefficient and slow

[kinnay]

Alright so I tried out github's organization features here. Organizations feel like a really nice way to group related repositories.

I'm a bit hesitant about creating an organization for the NintendoClients repo though. I think the reason is that it doesn't have a clear goal, unlike the playground.

For example, let's say that we want to implement Pia in this organization. Should we implement the game-specific layer of Pia in this organization as well, and if yes, for which games?

Also, while using one repository per service makes sense for server code, I don't think this is the right approach for client code. I mean, dauth.py is less than 300 lines of code. If we use one repository per service, the splitting becomes way too fine.

I'm thinking that the best choice might be to keep NintendoClients a monorepo and only create separate repositories when the effort to implement a new service is huge. So Pia would still get its own repository. Still, what about NPLN?

[kinnay]

it would be important to make the wiki into maybe it's own dedicated repository. That way other people can directly contribute, rather than making issues and hoping things get updated

We can do this even if we do not create an organization. There is a big downside though: it would break all existing links to the wiki. I guess I could replace all wiki pages with a big note that says that the wiki has moved, but this is not really ideal. It's just unfortunate that github doesn't support pull requests on wiki pages 😕

[jonbarrow]

I agree that is not ideal and GitHub lacking PR support for wiki's is a disappointing. I figured if things were moving to an org anyway it might be best to keep everything in one place

I'm a big fan of not having fragmented tools and information spread all around the internet, in my opinion that just makes information harder to find (which is also why I extended that offer on Discord)

I had actually be thinking about asking you for permission to re-host the wiki as a dedicated repo for some time now, so that way we at Pretendo can document our changes/findings more easily (obviously with the disclaimer that you are the original author)

[kinnay]

So, we are implementing dragons as well now (this was necessary for aauth). The main reason I've been hesitant to add more services and methods is the ban risk. But I kind of want to start implementing more services.

Of course I will implement the HTTP requests as accurately as possible, but this does not eliminate the ban risk. For example, even if I implement dragons correctly, the user can still provide an invalid elicense id. I have no idea if this causes a ban, but it does seem like something that Nintendo would be able to detect.

Maybe I should just put a big warning at the top of the readme and go on with it.

I think I want to worry less about the user anyway. More precisely, I'm thinking about:

• Removing games.py. I had to update all Switch game versions today because I've been neglecting this file. It's just an additional maintenance burden. It only covers a small selection of games anyway. I'm sure that motivated users can figure out the title id and version themselves.
• Removing some or all of example scripts. Again, it's an additional maintenance burden. I cannot test the GBG script anyway, because I haven't bought this game. Rather than providing a complete script, I probably want to provide short example snippets in the documentation instead. For example, I might show how to obtain an id token in one snippet, and show how to communicate with a NEX server in a separate snippet. Might keep the 3DS / Wii U scripts because there is no ban risk and Nintendo doesn't update these systems anymore (less maintenance burden).
• Worrying less about breaking backward compatibility. NintendoClients does not have a stable interface yet and even the readme says that at the top. Still, I'm often hesitant to make changes that are not backward compatible. If NintendoClients ever reaches the point where I believe that its interface is good enough I will probably use semantic versioning.

Then, maybe the goal of this repository would become providing a framework to communicate with Nintendo's servers, to:

1. Learn more about how they are designed. In my opinion, they are really well designed, and it is always helpful to see how such systems are designed in the real world. It is also fun to know how the games that I play work behind the scenes.
2. Facilitate cool projects such as mkdatabase.com.

I also removed the Pia code from this repository. Pia should definitely get its own repository if we ever manage to implement it correctly. I really want to focus on client-server communication here, so no Pia.

Again, just wanted to dump my thoughts.