Client disconnects if idle and server is behind Nginx #35

zorchenhimer · 2019-03-16T16:16:08Z

The chat client will disconnect if it is behind an Nginx reverse-proxy if no messages are sent within about a minute.

The client should send a PING to the server to keep the connection alive. The server needs to read the PING from the client, but I don't think it needs to respond with a PONG to keep the connection open.

MrAureliusR · 2022-03-27T05:55:16Z

I've just set MovieNight up behind nginx, and I'm having this issue. Is there a particular nginx option or header that needs to be added?

zorglube · 2022-03-27T12:49:56Z

Here is my Nginx conf, may you find something useful..

map $remote_addr $proxy_forwarded_elem {
	# IPv4 addresses can be sent as-is
	~^[0-9.]+$			"for=$remote_addr";
	
	# IPv6 addresses need to be bracketed and quoted
	~^[0-9A-Fa-f:.]+$	"for=\"[$remote_addr]\"";
	
	# Unix domain socket names cannot be represented in RFC 7239 syntax
	default				"for=unknown";
}

map $http_forwarded $proxy_add_forwarded {
	# If the incoming Forwarded header is syntactically valid, append to it
	"~^(,[ \\t]*)*([!#$%&'*+.^_`|~0-9A-Za-z-]+=([!#$%&'*+.^_`|~0-9A-Za-z-]+|\"([\\t \\x21\\x23-\\x5B\\x5D-\\x7E\\x80-\\xFF]|\\\\[\\t \\x21-\\x7E\\x80-\\xFF])*\"))?(;([!#$%&'*+.^_`|~0-9A-Za-z-]+=([!#$%&'*+.^_`|~0-9A-Za-z-]+|\"([\\t \\x21\\x23-\\x5B\\x5D-\\x7E\\x80-\\xFF]|\\\\[\\t \\x21-\\x7E\\x80-\\xFF])*\"))?)*([ \\t]*,([ \\t]*([!#$%&'*+.^_`|~0-9A-Za-z-]+=([!#$%&'*+.^_`|~0-9A-Za-z-]+|\"([\\t \\x21\\x23-\\x5B\\x5D-\\x7E\\x80-\\xFF]|\\\\[\\t \\x21-\\x7E\\x80-\\xFF])*\"))?(;([!#$%&'*+.^_`|~0-9A-Za-z-]+=([!#$%&'*+.^_`|~0-9A-Za-z-]+|\"([\\t \\x21\\x23-\\x5B\\x5D-\\x7E\\x80-\\xFF]|\\\\[\\t \\x21-\\x7E\\x80-\\xFF])*\"))?)*)?)*$"	"$http_forwarded, $proxy_forwarded_elem";

	# Otherwise, replace it
	default				"$proxy_forwarded_elem";
}

map $http_upgrade $connection_upgrade {
		default	upgrade;
		''		close;
}

server {
	listen 						80;

	server_name					[my domain].[tld];
	access_log 					/var/log/nginx/movienight.access.log;
	error_log 					/var/log/nginx/movienight.error.log;

	set 	$upstream 	xxx.xxx.xxx.xxx:8089;

	# Movie Night
	location / {
		proxy_pass			http://$upstream;
		proxy_http_version 		1.1;
		
		proxy_set_header		Host $host;
		proxy_set_header		X-Real-IP $remote_addr;
		proxy_set_header		Upgrade $http_upgrade;
		proxy_set_header		Connection $connection_upgrade;
		proxy_set_header		X-Forwarded-For $proxy_add_x_forwarded_for;
		proxy_set_header		Forwarded "$proxy_add_forwarded;proto=$scheme";
		
		proxy_redirect			off;
		proxy_next_upstream	error timeout invalid_header http_500 http_404;
		proxy_connect_timeout	2;
		proxy_intercept_errors	on;
	}

	# Movie Night Chat Websoket forward 
	location /ws {
		proxy_pass			http://$upstream;
		proxy_http_version		1.1;
		proxy_set_header		Host $host;
		proxy_set_header		Upgrade $http_upgrade;
		proxy_set_header		Connection $connection_upgrade;
		proxy_set_header		X-Forwarded-For $proxy_add_x_forwarded_for;
		proxy_set_header		Forwarded "$proxy_add_forwarded;proto=$scheme";
	}

	error_page 500 502 503 504 /error.html;
	location /error.html {
		return 				200 "<!DOCTYPE html><html><head><title>Movie Night</title></head><body style=\"background-color:black;\"><h1 style=\"color:grey;\">Server is down</h1></body></html>";
	}
}

zorchenhimer · 2022-03-27T15:01:36Z

@MrAureliusR Do you have any log output? Can you post your Nginx config (remove IP addresses if necessary)?

MrAureliusR · 2022-03-27T20:15:32Z

Here's what I came up with for the nginx config. I couldn't see anything in the readme about MovieNight requiring anything special, so I started off with just proxy_pass and then added a few other bits and pieces I've used when proxying other applications. The MovieNight log doesn't show anything when this happens -- it just shows the user leaving the chat.

After some more experimenting last night, I ended up adding

		proxy_connect_timeout 1d;
		proxy_send_timeout 1d;
		proxy_read_timeout 1d;

and that has fixed the issue, but that's kind of an ugly way to solve it.

server {
	server_name removed;

	location / {
		proxy_pass http://localhost:8089/;
		proxy_set_header Host $host;
		proxy_set_header Upgrade $http_upgrade;
		proxy_set_header Connection "upgrade";

		proxy_set_header X-Real-IP $remote_addr;
		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
		proxy_set_header X-Forwarded-Proto $scheme;

		add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload";
		add_header Referrer-Policy "same-origin";

		access_log /var/log/nginx/movienight.access.log;
		error_log /var/log/nginx/movienight.error.log;
	}

    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/removed/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/removed/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}
server {
    if ($host = removed) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


	server_name removed;

	listen 80;
    return 404; # managed by Certbot


}

zorchenhimer · 2022-03-27T20:32:05Z

Hmm, the only real differences I have from your config are that I separated out websocket connections into their own block and set the proxy_http_version to 1.1 in the websocket block:

location / {
        proxy_pass http://movienight;
}

location /ws {
        proxy_pass http://movienight;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}

Do you have anything in your Nginx logs?

MrAureliusR · 2022-03-28T00:48:35Z

Would that actually make a difference?

…

On Mar. 27, 2022, 16:32, at 16:32, Nick ***@***.***> wrote: Hmm, the only real differences I have from your config are that I separated out websocket connections into their own block and set the `proxy_http_version` to `1.1` in the websocket block: ``` location / { proxy_pass http://movienight; } location /ws { proxy_pass http://movienight; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "Upgrade"; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } ``` -- Reply to this email directly or view it on GitHub: #35 (comment) You are receiving this because you were mentioned. Message ID: ***@***.***>

joeyak · 2022-03-28T03:33:08Z

I think what happened was in the removal of wasm I forgot to convert this line over to the javascript, so now there's nothing to keep the websocket connection open
80be381#diff-1a75e4f727f20d549964a1872e088c634844e0b3aa68233bf822ab6a125ffe87R264

joeyak · 2022-03-28T04:12:37Z

Turns out nginx times out websocket connections at 60 seconds http://nginx.org/en/docs/http/websocket.html

joeyak · 2022-03-28T04:19:25Z

@MrAureliusR I worked with someone else to check the fix in PR #182. I'm pretty sure this is solved we can reopen the issue if you still have it

joeyak · 2022-03-28T04:22:07Z

Or not, seems like they didn't change the nginx config back to the original setup that broke it

MrAureliusR · 2022-03-28T04:51:26Z

Mine is set up as a docker container. If there's a docker image I can pull and test quickly I'd be happy to do so.

…

On Mar. 28, 2022, 00:22, at 00:22, joeyak ***@***.***> wrote: Or not, seems like they didn't change the nginx config back to the original setup that broke it -- Reply to this email directly or view it on GitHub: #35 (comment) You are receiving this because you were mentioned. Message ID: ***@***.***>

CptPie · 2022-03-28T22:44:54Z

Mine is set up as a docker container. If there's a docker image I can pull and test quickly I'd be happy to do so.

I am not 100% certain on this one but you SHOULD be able to just git pull and then restart the container (maybe rebuild the container to be save).

zorchenhimer added bug Something isn't working hardening Changes that are not new features, but benefit the developer labels Mar 16, 2019

zorchenhimer self-assigned this Mar 16, 2019

zorchenhimer closed this as completed in 80be381 Mar 16, 2019

joeyak mentioned this issue Mar 17, 2019

Add *user is typing* notifications like on rabbit and discord #36

Closed

zorchenhimer reopened this Mar 27, 2022

joeyak mentioned this issue Mar 28, 2022

Fix ping removal #182

Merged

joeyak closed this as completed in #182 Mar 28, 2022

joeyak reopened this Mar 28, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Client disconnects if idle and server is behind Nginx #35

Client disconnects if idle and server is behind Nginx #35

zorchenhimer commented Mar 16, 2019

MrAureliusR commented Mar 27, 2022

zorglube commented Mar 27, 2022 •

edited

Loading

zorchenhimer commented Mar 27, 2022

MrAureliusR commented Mar 27, 2022 •

edited

Loading

zorchenhimer commented Mar 27, 2022 •

edited

Loading

MrAureliusR commented Mar 28, 2022 via email

joeyak commented Mar 28, 2022

joeyak commented Mar 28, 2022

joeyak commented Mar 28, 2022

joeyak commented Mar 28, 2022

MrAureliusR commented Mar 28, 2022 via email

CptPie commented Mar 28, 2022

Client disconnects if idle and server is behind Nginx #35

Client disconnects if idle and server is behind Nginx #35

Comments

zorchenhimer commented Mar 16, 2019

MrAureliusR commented Mar 27, 2022

zorglube commented Mar 27, 2022 • edited Loading

zorchenhimer commented Mar 27, 2022

MrAureliusR commented Mar 27, 2022 • edited Loading

zorchenhimer commented Mar 27, 2022 • edited Loading

MrAureliusR commented Mar 28, 2022 via email

joeyak commented Mar 28, 2022

joeyak commented Mar 28, 2022

joeyak commented Mar 28, 2022

joeyak commented Mar 28, 2022

MrAureliusR commented Mar 28, 2022 via email

CptPie commented Mar 28, 2022

zorglube commented Mar 27, 2022 •

edited

Loading

MrAureliusR commented Mar 27, 2022 •

edited

Loading

zorchenhimer commented Mar 27, 2022 •

edited

Loading