Add HTTP server support

[jukkar]

This is based on the work done in #59669. I created a new PR as the original PR was kind of messy and long.

I fixed the tests so that they now use picolibc. Also the sample is tweaked to compile properly in native_sim.
Fixed also Posix support (basically removed it) so that it compiles properly with socket API.

[cfriedt]

These should probably not go in this header.

[jukkar]

Yes, agreed. I think we need to create internal http server header file for these.

[pdgendt]

I've created a CoAP server implementation (#64265) inspired by the work that was done in the previous PR.
I did leave out the secure variant for now, since this way of doing it, forces it to either secure or non-secure. But no option to support both.

[jukkar]

IMHO we need to allow supporting both at the same time so some tweaking is needed here. Initially I was thinking that application could do the socket initialization for TLS which would move the https burden from the lib to the application needing that support. For example, when registering the service, user could supply a function pointer that the server could then call to initialize the socket to use TLS.

[pdgendt]

I have been through the same process, I was working on a PoC for my CoAP server where the service is either secure or not, something like:

#define COAP_SERVICE_DEFINE(_name, _host, _port, _flags) /* Put NULL in the DTLS data */
#define COAP_SERVICE_DEFINE_SECURE(_name, _host, _port, _flags, _dtls_cfg) /* Some struct with DTLS info is provided */

The server implementation could then verify the secure data struct on how to initialize the sockets.

[jukkar]

• Updated according to comments
• Squashed couple of commits together, the history is a bit messy so did not dive too deep into this
• Added initial https support (does not work yet), I am proposing that TLS socket is done outside of the http_server library in order to avoid TLS complexities involved. This in mind I added a socket setup callback that the application can set when defining a service, and the library will then call it when needed

[pdgendt]

• Added initial https support (does not work yet), I am proposing that TLS socket is done outside of the http_server library in order to avoid TLS complexities involved. This in mind I added a socket setup callback that the application can set when defining a service, and the library will then call it when needed

While I like the customization level, moving the TLS complexities outside of the library seems like a step back to me. I would prefer, as a user, that the subsystem provides the implementation for a certain configuration. I see some benefits that way:

• The same implementation across projects
• Testable subsystem for the security features, and tracking of vulnerabilities
• It just works ™
• Avoid bugs/issues for a plethora of different uses of the socket_setup function

If the subsystem would expose a socket_setup function that is re-usable, this would already be a big plus. Some MACRO magic could hide most details for example.

Just my 2 cents :)

[jukkar]

If the subsystem would expose a socket_setup function that is re-usable, this would already be a big plus. Some MACRO magic could hide most details for example.

Perhaps this is a way to go. So we could provide a default way to setup the TLS but if needed application could override that and do things its own way.

[jukkar]

Worked on supporting multiple clients. Did not get it fully working yet.

[jukkar]

This is now somewhat usable as a library. More than one client can be supported at the same time.

Still missing:

• https support
• dynamic resources
• overall cleanup of the code

[jukkar]

Changes:

• fixed the multiple client support, client struct needed proper cleanup after connection was closed
• preparation for https, removed the socket callback, user needs to supply the sec_tag list and the TLS socket will be created using that info

[jukkar]

HTTPS works now with attached certs which were copied from echo-server sample.

[jukkar]

I am wondering about the dynamic resources, how they are defined and what kind of API between application and lib would work best. Any suggestion / ideas for that?

[cfriedt]

I am wondering about the dynamic resources, how they are defined and what kind of API between application and lib would work best. Any suggestion / ideas for that?

As in REST APIs? It would be a different kind of handler with a different "detail" pointer.

[jukkar]

As in REST APIs? It would be a different kind of handler with a different "detail" pointer.

Yes, I was thinking that too and did some experiments. Will send changes after get something working.

[jukkar]

Updated according to comments.

[jukkar]

• Minor refactoring done, Robert split the http_server.c into three parts, one for core functionality, one for HTTP/1 processing and one for HTTP/2
• HTTP/1 header parsing is handled properly even if header is received in smaller pieces

[jukkar]

@cfriedt @pdgendt @nxpadamm @MeisterBob and all the others in the review list.
Robert has done great job of fixing and implementing remaining known issues, also the commits are now squashed as the history was too messy again, so we believe the PR is ready for a merge. Please take a look at the code and if possible try it yourself and give feedback.

I have the original branch (with 75 commits) as a backup if we need it for anything.

[rlubos]

fixing and implementing remaining known issues

Just to be clear - not all remaining/known issues, but the implementation is quite functional already and the PR is already pretty large. Further improvements/fixes can/will be submitted in separate PRs.

[cfriedt]

@Emna-Rekik 🪁🎉🥳

[jukkar]

• updated according to comments
• fixed sample readme file as it was describing things that were not correct any more

[jukkar]

Related to discussion in #70817, @fabiobaltieri would you mind force merging this, there are binary certificate files that cause CI to fail.

[fabiobaltieri]

Thanks @jukkar I don't have permissions to do it myself though, @carlescufi can you take care of this one?

[carlescufi]

@jukkar shouldn't we wait for @rlubos' approval here?

[jukkar]

@jukkar shouldn't we wait for @rlubos' approval here?

Well, Robert is one of the authors of these patches so I am not sure if that would change much, but if you want we can wait for that.

[carlescufi]

@jukkar shouldn't we wait for @rlubos' approval here?

Well, Robert is one of the authors of these patches so I am not sure if that would change much, but if you want we can wait for that.

Right, you are right. No need to wait in that case, I will merge it.

[kartben]

@rlubos @jukkar As a follow-up to this PR, could you please look at adding minimal docs for this new feature?
Right now it doesn't even have Doxygen groups for the new headers, nor does it have any entry in the documentation (could be just an empty "placeholder" for listing the Doxygen group(s) under an API Reference section). Bonus points for using zephyr:code-sample:: in the code sample so that it cross-references the API. Thank you!

[jukkar]

As a follow-up to this PR, could you please look at adding minimal docs for this new feature?

Yes, that is in the todo list, we will address this before 3.7.