Working Server Example

[danomagnum]

I've been working on getting the server working and I've got a version of it working on the server branch of my fork here.

There is still quite a bit to implement to get it production ready, but as a proof-of-concept it is working. I've been (manually) testing it using ignition and pyopcua.

I tried to keep as much of the new code in /examples/server/ as I could.

What (mostly) works:

• Browse
• Read Attribute (value, type)
• Write Attribute (value)
• Create Subscription
• Create Monitored Items
• Publish (values only. timed only - not event based/ on change)

I would like to contribute this back to the project, but I've got to do quite a bit of refactoring first and would like some recommendations on what the refactored code should look like before I start on that so that I don't have to refactor it again to match what you all are looking for in the server code.

As an example, one thing that will have to change is I am ignoring the concept as nodes as much as possible - I've got a map[string]any providing the backed and only using string node IDs as the keys to the map. This is a lot more convenient for the actual use case I've got that prompted me to start working on this so I'd like to keep that functionality in the end, but I think I should be able to make a pseudo-node that would hold the map and provide the references, etc... to make it invisible to the server. Maybe.

Anyway, like I said I'm really looking for feedback, suggestions, and recommendations on what you'd like to see. Obviously my use case is really what I need to achieve but if I can make it work for merging back in that would be a win-win.

[danomagnum]

I ended up splitting the address space up by namespace and made a namespace interface so different back-ends can be added at the namespace level. I've ported the browse and read/write attribute handlers back into the main server and it is working for regular nodes and my map-based "nodes".

The pub/sub stuff is next, but I'll have to think about the best way to do that for a bit before I start moving it back in.

[magiconair]

Wow 🥇 Thank you @danomagnum !!! That might be the push to get us off the ground with the server. I'll have a look.

[magiconair]

CC: @kung-foo

[danomagnum]

So I've got quite a bit of it working now. I've re-integrated it back into the server module of the library - nothing in the example folder except for an actual example using the library to host a server now.

Assuming the simplest use case is the most common, I think it's pretty usable at this point.

I tested it now against ignition's UA client, pyopc's client, and UaExpert. They're all reading/writing/subscribing to data and working correctly.

I'm sure there are still bugs to work out of course.

Once you get a chance to look at it let me know and I'll get it merged back up to the latest master client revision and I can do a pull request.

Here's the current status of all the services in the ua spec:

Discovery Service - unchanged (assumed working)
	Find Servers
	Find Servers On Network
	Get Endpoints
	Register Server
	Register Server 2
Secure Channel Service - unchanged (assumed working)
	Open Secure Channel
	Close Secure Channel
Session Service - Unchanged (assumed working)
	Create Session
	Activate Session
	Close Session
	Cancel
Node Management Service - Not implemented from the UA api - you can do it through the go function calls though.
	Add Nodes
	Add References
	Delete Nodes
	Delete References
View Service
	Browse - Working.  Does not support partial browse/BrowseNext.
	BrowseNext - Not implemented
	TranslateBrowsePathsToNodeIds - Not implemented
	RegisterNodes - Not implemented
	UnregisterNodes - Not implemented
Query Service - Not implemented
	QueryFirst
	QueryNext
Attribute Service
	Read - Working
	HistoryRead - Not implemented
	Write - Working
	HistoryUpdate - Not implemented
Method Service - Not implemented, but it would be easy enough to add by  adding a Call method to the Namespace interface.
	Call
Monitored Item Service
	Create Monitored Items - Working
	Modify Monitored Items - Working
	Set Monitoring Mode - Working
	Set Triggering - Not implemented
	Delete Monitored Items - Working
Subscription Service
	Create Subscription - Working
	Modify Subscription - Working
	Set Publishing Mode - Not implemented
	Publish - Not implemented
	Re-Publish - Not implemented
	Transfer Subscriptions - Not implemented
	Delete Subscriptions - Working

[magiconair]

The thing I got stuck at was to get the basic node structure in the prosys browser. The Root, Objects, Types, Views and so forth. Maybe you can get this to work and then I'm all for merging this back to main and marking this experimental. But I'd like people to play with this and drive this forward. So once we have a basic programming model where someone can write a server and make it do something then we can iterate on this.

gopcua server

standard opcua server

[magiconair]

I think you need to implement browsing for this.

[danomagnum]

So browsing (edit: partially - no continuation/browsenext) works, but you've got to have all the node references defined for it to know what to return as the browse results.

If I manually set them up, they start showing up.

// add the namespaces to the server, and add a reference to them
	root_ns, _ := s.Namespace(0)
	root_obj := root_ns.Root()

	// add the "Types" folder
	tf := root_ns.Node(ua.NewNumericNodeID(0, 86))
	root_obj.AddRef(tf)
	dtf := root_ns.Node(ua.NewNumericNodeID(0, 90))
	tf.AddRef(dtf)
	base_type := root_ns.Node(ua.NewNumericNodeID(0, 24))
	dtf.AddRef(base_type)
	bool_type := root_ns.Node(ua.NewNumericNodeID(0, 1))
	ByteString_type := root_ns.Node(ua.NewNumericNodeID(0, 15))
	DataValue_type := root_ns.Node(ua.NewNumericNodeID(0, 23))
	DateTime_type := root_ns.Node(ua.NewNumericNodeID(0, 13))
	DiagnosticInfo_type := root_ns.Node(ua.NewNumericNodeID(0, 25))
	Enumeration_type := root_ns.Node(ua.NewNumericNodeID(0, 29))
	Uint32_type := root_ns.Node(ua.NewNumericNodeID(0, 7))
	dtf.AddRef(bool_type)
	dtf.AddRef(DataValue_type)
	dtf.AddRef(ByteString_type)
	dtf.AddRef(DateTime_type)
	dtf.AddRef(DiagnosticInfo_type)
	dtf.AddRef(Enumeration_type)
	dtf.AddRef(Uint32_type)


	root_obj = root_ns.Objects()

	server_obj := root_ns.Node(ua.NewNumericNodeID(0, 2253))
	root_obj.AddRef(server_obj)

I only did a handful of the references to see if it worked or not. There is an absolute ton of references that are needed to get everything filled in properly. I'll have to take a look at how nodes_gen is being generated to see if we can build those references automatically.

[danomagnum]

I made an adjustment to the generation file and while it's not perfect yet it's browsing the pre-defined nodes automatically now. Need to do some more investigation on why they are showing up as X's instead of folder icons. There are some loops in the references too that need to be sorted out.

[magiconair]

How about merging your changes on top of our server branch and giving you access to this repo? Then we can collaborate on your changes and I can help with refactoring. The collaboration on branches is something I really miss from Gerrit unless I'm missing something. I'll send you an invite.

[magiconair]

To get the full nodes generated we should probably import a NodeSpec2.xml file and generate the code from that. Sooner or later we need to add support for importing a node set anyway.

[magiconair]

@danomagnum I've sent you an invite. You should use the second one since I've canceled the first one.

[danomagnum]

Invite accepted.

I'll try and get the changes merged properly onto the server branch in the next couple days and I'll let you know. I will probably bring in the changes from the main branch too, just so the server branch gets caught back up.

I've been away from this project for the holidays but intend to pick it back up in the next week or two. Before the break I was able to get a lot (but still not all) of the auto-import from xml working using the PredefinedNodes.xml file (which looks like it is generated by the opc-ua dotnet project from the nodeset2.xml file?). At one point the generated .go file it created was too large and the compiler choked on it so I had to refactor how it was generating the cross-references to re-use references instead of generating them anew for each node.

The strangest thing going on right now is that the FolderType reference is showing up as a child item when browsing, even though as far as I can tell there isn't any difference between the packets gopcua is sending and the packets other similar opc servers send. So far getting the predefined nodes more correct has fixed this kind of thing so that's where I'm betting the problem is.

As you suggest, It may probably be easier to get everything correct if we start with the nodeset2.xml file directly instead of going through the generated xml file.

[magiconair]

Maybe we can embed the Nodeset2.xml file and parse it on startup.

[danomagnum]

Switching to the nodeset2 xml file was a good move since there is a schema definition file available and I was able to auto-generate the go structs for it using xgen which saved a lot of time parsing it. I feel much more confident the references are correct using this method.

It is embedded in the schema sub package and is now loading on server startup instead of using the generated file.

Unfortunately, the strange behavior with the FolderType showing up under every folder during browse with some clients remains.

I merged everything into the server branch of the main repo.

So far I had been testing without encryption so I could easily wireshark the packets. I tried using encryption today and that was not an immediate success so there may be some work to do there also.

[magiconair]

Awesome. I'll have a look

[magiconair]

Hmm, it doesn't compile. Did you push all of your changes?

frank@piet2 ~/s/g/g/opcua (server)> make
go test -count=1 -race ./...
# github.com/gopcua/opcua/uasc
package github.com/gopcua/opcua/uasc
	imports github.com/gopcua/opcua/uatest
	imports github.com/gopcua/opcua/server
	imports github.com/gopcua/opcua/uasc: import cycle not allowed in test
FAIL	github.com/gopcua/opcua/uasc [setup failed]
# github.com/gopcua/opcua
./client.go:766:54: cannot use func(v interface{}) error {…} (value of type func(v interface{}) error) as uasc.ResponseHandler value in argument to c.SecureChannel().SendRequest
./client.go:879:77: cannot use func(v interface{}) error {…} (value of type func(v interface{}) error) as uasc.ResponseHandler value in argument to c.SecureChannel().SendRequest
./client.go:942:66: cannot use h (variable of type func(interface{}) error) as uasc.ResponseHandler value in argument to c.sendWithTimeout
./client_sub.go:181:88: cannot use func(v interface{}) error {…} (value of type func(v interface{}) error) as uasc.ResponseHandler value in argument to c.SecureChannel().SendRequest

[magiconair]

I'm here.

commit 1ddfa35 (HEAD -> server, origin/server)
Author: danomagnum <[email protected]>
Date:   Wed Jan 3 21:01:03 2024 -0600

    Added server info to the readme.md

[magiconair]

Which timezone are you in? Just for my info. I'm Europe/Stockholm (UTC+1)

[danomagnum]

I see. I haven't been building everything with make, just doing go run in the specific examples I was testing. Looks like when I merged in the main to the server branch I didn't get the change to the ua.Response interface everywhere somehow. Hopefully all taken care of now.

I'm in US central (UTC-6).

[magiconair]

Getting test timeouts

Listening on 0.0.0.0:4841
--- FAIL: TestNamespace (10.08s)
panic: opcua: timeout [recovered]
	panic: opcua: timeout

goroutine 54 [running]:
testing.tRunner.func1.2({0x102fb5c40, 0xc00000e408})
	/Users/frank/sdk/go1.21.0/src/testing/testing.go:1545 +0x274
testing.tRunner.func1()
	/Users/frank/sdk/go1.21.0/src/testing/testing.go:1548 +0x448
panic({0x102fb5c40?, 0xc00000e408?})
	/Users/frank/sdk/go1.21.0/src/runtime/panic.go:920 +0x26c
github.com/gopcua/opcua/uatest.NewPythonServer({0x102ebe80b, 0xc})
	/Users/frank/src/github.com/gopcua/opcua/uatest/py_server.go:46 +0x124
github.com/gopcua/opcua/uatest.TestNamespace(0xc0002149c0)
	/Users/frank/src/github.com/gopcua/opcua/uatest/namespace_test.go:18 +0x3c
testing.tRunner(0xc0002149c0, 0x102ffd420)
	/Users/frank/sdk/go1.21.0/src/testing/testing.go:1595 +0x198
created by testing.(*T).Run in goroutine 1
	/Users/frank/sdk/go1.21.0/src/testing/testing.go:1648 +0x5dc

[danomagnum]

Looks like the integration server python file was changed after the initial server branch was created to use port 4841 instead of 4840. I set it back and it should be OK now.

[megakoresh]

Heya, so just wondering - is there anyone working on this feature rn? There is a draft PR for it but that's from a different branch. But the code on the server branch looks very close to finished, at least at first glance. So what's the status on this? Any progress?

[danomagnum]

I haven't worked on it in a little bit because it's (the server branch) mostly implemented enough for my needs. I've had it running in a non-critical production application for a couple months now with no issues.

I didn't ever comment about it in this issue, but I did fix the FolderType bug and everything looks pretty good now when browsing from a client. Read, Write, Subscribe, and Browse all work, so it's pretty functional.

The two things I was working on when I got pulled in a different direction were:

• Getting the integration tests set up using gopcua for both client and server (in addition to rather than instead of the python opc server it is currently using).
• Getting the crypto working properly.

I was able to change a couple lines in the code and get specific encryption/signing methods to work, but I wasn't having much luck picking up on the correct method programmatically.

If you need a server, you should be able to specifically pull the server branch - it's up to date with master except for some readme changes.

I'm open to issuing a pull request to merge it into the main branch if everyone else is good with it.

[megakoresh]

I would love for that branch to be merged. If you know of some issues with it, then why not just merge it to main and immediately create the issues so they can be tracked? And if the server is already working in production for you, then IMO it's good to go as a first version. Better than not having it at all. Just put a usage example to the examples folda and if the crypto is not fully working, it can just be marked with panic("unimplemented")

[megakoresh]

@magiconair is this something that can be done? I am trying to advocate for use of this library over the python one because of the performance, but its hard without a working server implementation

[JessevGool]

Would love to have this merged as well! Would also make it so that more people are aware of the server itself which might just help with solving the open issues

[danomagnum]

Just FYI, I messed something up when I put the pull request together. The version in my repo is working correctly, but I need to re-merge it into the server branch of this repo and re-issue a pull request.

[danomagnum]

Pull request is back live. Now #737