socketio service to service communication via message queue

[arun6894]

1. when we use client manager in socket service, client to server will communicate via message queue and the same applies for a server to a client via a message queue?

2. We have a requirement to communicate the socket service(python3) with other service(python2.7). Is there a way to push only data to kafka/redis message queue so that other service can able to access it and at the same time if other service places some data to message queue, socket service should be able to pick it and emit client ?

3. Is there a better way to communicate two services with different python versions via message queue and emit messages from socket service to client?

client <-- > socket service(python3) <--> (any message queue) <--> Other service(python2.7)

Please confirm if we can use any feature of python-socketio to achieve this. If not is it possible in any other way like maintaining a connection pool for both services of their own python versions and communicating via that?

[miguelgrinberg]

1. You are mistaken. The message queue does not change anything. Clients communicate directly with the server, and the server communicates directly with clients. The message queue is used internally when you have more than one server, as a way to coordinate operations such as broadcasts. There is nothing related to the message queue that can be used from your application.
2. Socket.IO cannot help with communication between two processes on the server-side, unless one process is a Socket.IO client and the other is a Socket.IO server. If you need to use a message queue as a way to pass messages between the two process that is fine, but this package cannot help with that, you have to implement this yourself.
3. I don't understand this question. As I said, the use of a queue in this package is an internal implementation detail. There is no queue functionality that is exposed to the application.

[srinivasnm471]

Hi @miguelgrinberg
I am using redis as a messaging queue to coordinate between multiple servers, to explain the topology I have socketio client cl-1 connected to socketio server s1, similarly I have a connection b/w cl-2 and s2, I had to publish a message to both cl-1 and cl-2, so I used external_sio from redis to send messages to both my clients, I was able to see the messages, but I would want to get a reply from cl-1 and cl-2 to external_sio (sender/publisher), is this possible?

Like you mentioned communication can be b/w a socketio server and socketio client but can my redis publisher also recieve a message from clients?

[miguelgrinberg]

I would want to get a reply from cl-1 and cl-2 to external_sio (sender/publisher), is this possible?

No. Only servers receive messages from clients. If you want to pass a message from the server to the external process, you have to implement that yourself.

Like you mentioned communication can be b/w a socketio server and socketio client but can my redis publisher also recieve a message from clients?

This package doesn't do this. But nothing prevents you from using any inter-process communication method to pass data between these two processes.

[srinivasnm471]

@miguelgrinberg
I am using the program mentioned in document to publish data to redis:

external_sio = socketio.RedisManager('redis://127.0.0.1:6379', write_only=True)
external_sio.emit('message', data={'message': 'sample_program'})

This inturn sends data to socketio client (using socketio server that is configured to listen to redis). In response to this message my client generates a response and sends it to server. My requirement was to capture this response on the redis publisher. I used thread that redis manager offered to listen to the message that server got.

external_sio._thread()

added a print in _thread method to check if I was able to listen to the message, I was able to see the message till the program passed it to self._handle_emit(data) function, from where it expected namespace which caused program to exit, is there any other way to retrieve the message that client sent to redis publisher?

[miguelgrinberg]

My requirement was to capture this response on the redis publisher.

I mentioned several times already that this package does not support this. The purpose of the RedisManager is to emit, not to receive. You will receive messages from the client only in the server process. From there, you are free to communicate with your other process using any mechanism that you like, including a Redis pub/sub mechanism (on your own channel, not on the channel that Socket.IO uses).

added a print in _thread method to check if I was able to listen to the message, I was able to see the message till the program passed it to self._handle_emit(data) function

The _handle_emit() method handles emits from server to client. You don't seem to believe me when I tell you that there is no logic that supports passing client messages to the external process. There really isn't anything, this isn't a use case that I ever intended to support.

[srinivasnm471]

@miguelgrinberg
Thanks, will proceed in the mentioned direction.