Add API on NiceHashMiner

[delawr0190]

To help with the integration of NiceHashMiner into https://foreman.mn, would it be possible to add a simple API on NiceHashMiner that will return the miner that's currently running and the API port? Something like:

{
  "minerType": "phoenix",
  "apiPort": 4000
}

With the ports being 1-up when there's a bind, it's difficult to find the miner that's being ran by nicehashminer. A simple HTTP API would be great to cut down the discovery process.

[vasyapupkin1]

maybe try \127.0.0.0:4000 \127.0.0.0:4001.... during minig and parse it?

[delawr0190]

The issue with that is that you have to determine the miner that’s running from the api output, though. And some of them are drastically different. And I’ve seen ports as high as 4005. So the discovery process of finding miners can end up being a real pain.

A simple api to at least point in the right direction of the miner’s api would be a tremendous help.

[grichter]

If you're weird like me and run 8 cards that are different enough that you end up with 6 different miners running just scanning a range of ports and trying to guess what miner is on each would be obnoxious.... and then there's the respawns due to algo profitability changes.

[S74nk0]

@delawr0190 You could achieve this with MinerReservedPorts. I think this is exactly what you need.

So what you would do is reserve different ports for different algorithms for a specific miner plugin. Each miner plugin (at least the ones implemented by the NiceHash team) has this internal setting MinerReserverPorts.json. So what I would suggest is to take your own range and split it up by each miner and for each miner reserve a specific range lets say at least 5 ports per algorithm. So lets say you would do this for ClaymoreDual14+ and Phoenix for DaggerHashimoto this is what you would do:

ClaymoreDual14+ file
nhm_path\miner_plugins\ClaymoreDual14+\internals\MinerReservedPorts.json:

{
  "use_user_settings": true,
  "algorithm_reserved_ports": {
        "DaggerHashimoto": [4200, 4201, 4202, 4203, 4204],
    }
}

Phoenix file
nhm_path\Debug\miner_plugins\Phoenix\internals\MinerReservedPorts.json:

{
  "use_user_settings": true,
  "algorithm_reserved_ports": {
        "DaggerHashimoto": [4100, 4101, 4102, 4103, 4104],
    }
}

Now nhm will reserve these ports. This way you will know that range [4200-4204] is miner ClaymoreDual14+ and algorithm DaggerHashimoto.

If this doesn't completely solve your problem please elaborate. Otherwise I suggest we close this issue.

[delawr0190]

@S74nk0 Would it be possible for the ranges to be assigned by nhm in the default configuration? The solution that you proposed sounds reasonable, and I do believe that it would address my request. However, from the perspective where I'm an external service and I'm trying to integrate with nhm, this puts a good amount of configuration on the user. To integrate with nhm, I either have to walk across ports and try to determine where miners are running and what type they are. Or I have to ask users to do a good amount of configuration to nhm so that we can have an easier integration.

Our primary objective is ease of use for our users. If it came down to us walking ports and finding miners or asking users to get deeper into nhm and learning how to configure each miner, I'd prefer to walk ports. Saves user intervention.

An API that exposes the miner and the API port would be great. That would require no user intervention, and it will tell Foreman exactly what needs to be queried.

[S74nk0]

@delawr0190 We can automate this and there would be no user intervention needed. An external tool that would walk up all the miners (integrated and externally installed) to reserve ports and it would generate a JSON config for your service. The config should tell you the reserved ports of supported algorithms of each miner plugin and of course it will guarantee that no ports overlap.

We already have a similar tool to automate external plugin packaging in Tools. We can make a tool that would setup the internal settings and generate a config for your service. I propose this solution because it would be fastest to implement at the moment (thanks to the modular design of the new plugin system). I guess it is better to have something sooner than later. We are actively changing and refactoring the nhm core to make it more modular (just like the miner plugin system) and until we get there I would wait on adding new features.

What do you say on an external tool that automates and sets up the plugins? Will this do it for now?

[delawr0190]

@S74nk0 I really appreciate you working with me through this request.

Some context regarding our product: we have an agent that gets installed into the user's LAN and then, from our dashboard, they add miners that they wish to query and centralize to our dashboard.

For the tool that you're proposing above, would the JSON configuration be the same across all instances of nhm, or would it possibly be different if a user were running nhm on host 1 with certain plugins and nhm on host 2 with different plugins. I think that you're saying that the JSON would have port ranges assigned by algorithm - I just want to make sure that I'm understanding what you're saying. Any way you could provide a little JSON sample?

Will the JSON include the miner type, too, that could be running for that algorithm?

If we can get down to per miner type port ranges, it would cut down my discovery process substantially.

[S74nk0]

@delawr0190 We can work together on the config format. But if I understand correctly this would be sufficient for your case:

[
{
  "uuid": "3d4e56b0-7238-11e9-b20c-f9f12eb6d835",
  "name": "XmrStak",
  "algorithm_reserved_ports": {
        "CryptoNightR": [4321, 5894],
    }
},
{
  "uuid": "78d0bd8b-4d8f-4b7e-b393-e8ac6a83ae76",
  "name": "ClaymoreDual14+",
  "algorithm_reserved_ports": {
        "DaggerHashimoto": [4100, 4101, 4102, 4103, 4104],
        "DaggerHashimoto+Decred": [4110, 4111, 4112, 4113, 4114],
    }
},
{
  "uuid": "1b7019d0-7237-11e9-b20c-f9f12eb6d835",
  "name": "GMinerCuda9.0+",
  "algorithm_reserved_ports": {
        "ZHash": [5100, 5101, 5102, 5103, 5104],
        "Beam": [5110, 5111, 5112, 5113, 5114],
        "GrinCuckatoo31": [5120, 5121, 5122, 5123, 5124],
        "CuckooCycle": [5130, 5131, 5132, 5133, 5134],
        "GrinCuckarood29": [5140, 5141, 5142, 5143, 5144],
    }
}
]

It would look something like this for example and array of miner plugins with all supported algorithms and their reserved ports. Now the settings would differ from machine to machine depending on the rig hardware (AMD only, NVIDIA only, mixed rig)... But the format would be the same for every plugin. Now the hacky part would be tailing the log file to check the switch and when to scan for ports.
I would suggest you look for:
Internal plugin:

[2019-07-18 10:40:32] [INFO] [MiningManager] Stop Old Mining: (EMTPY)
[2019-07-18 10:40:32] [INFO] [MiningManager] Start New Mining : (X16R_CCMinerTpruvot(GPU-3a58a23b-efad-f03f-fc6f-ae0d281d59cd,GPU-4c6bf789-bc08-469e-8f92-e47d342780c6))
[2019-07-18 10:40:32] [INFO] [MiningManager] No change : (EMTPY)

External plugin:

[2019-07-18 10:44:20] [INFO] [MiningManager] Stop Old Mining: (EMTPY)
[2019-07-18 10:44:20] [INFO] [MiningManager] Start New Mining : (X16R_5532d300-7238-11e9-b20c-f9f12eb6d835(GPU-4c6bf789-bc08-469e-8f92-e47d342780c6))
[2019-07-18 10:44:20] [INFO] [MiningManager] No change : (EMTPY)

So it is ALGO_UUID(devices) so ALGO and UUID are your keys to know what to scan. I admit that the log tailing is a (horrible) hack and not the best approach but until we refactor the mining core I would wait on adding new features. After the re-write of mining core we will add an API.

[delawr0190]

@S74nk0 I think that configuration looks great.

I worry about the user involvement, though, from our end still. This changes our deployment model a bit: our agents now need to be ran locally with nhm, and that's different than our deployment posture at the moment. We generally require only one agent within the LAN, which limits our ability to do any log parsing.

This will also require that users runs a tool to generate this, too, for every miner, correct?

If there is at least an API on the horizon, we will most likely wait until then. We are okay doing port scanning at the moment, but it's going to become problematic if the number of miners that nhm supports continues to expand, and it probably will.

For the short term, would it be possible to unify at least the range starting port down to 4000? I've noticed that 4000 seems to be the current starting port for the range, but there's some error handling in

NiceHashMiner/src/NiceHashMiner/Configs/Data/GeneralConfig.cs

Line 306 in b5c7037

ApiBindPortPoolStart = 5100;

that seems to bump it back down to 5100 if the range is too high. Could the range checking at least rebound it to 4000?

This would let us probe for ports (4000 - 4005) * number of miners supported. Less user involvement, and we don't need to have users run tools or run their agents local to nhm. We are okay for now - just an API on the horizon would be greatly appreciated.

[S74nk0]

@delawr0190 I understand. Don't worry about that line it should be 4000 that gets called only on start and when user passes the range settings. The start port is *4000 by default.

We will provide an API when time permits. Keep issue open for now