OS: Ubuntu 20.04 (all updates installed)
Issue: When I upgraded from 6.4 to 7 using the repositories the resulting document server became non-functional.
Has anyone had this happen to them? I grabbed all the logs. What should I be looking for?
NGiNX Logs:
Error.log: 2022/01/22 00:07:47 [emerg] 352#352: chown("/dev/null", 33) failed (1: Operation not permitted)
Error Log in onlyoffice:
2022/01/20 13:41:30 [error] 3990389#3990389: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.40, server: , request: "GET /7.0.0-132/doc/1016363361/c/info?t=1642686090224 HTTP/1.1", upstream: "http://127.0.0.1:8000/doc/1016363361/c/info", host: "192.168.1.12", referrer: "http://192.168.1.12/7.0.0-132/web-apps/apps/spreadsheeteditor/main/index_loader.html?_dc=7.0.0-132&lang=en&frameEditorId=iframeEditor&parentOrigin=http://192.168.1.41"
2022/01/20 13:41:32 [error] 3990391#3990391: *6 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.40, server: , request: "GET /7.0.0-132/doc/1016363361/c/info?t=1642686092426 HTTP/1.1", upstream: "http://127.0.0.1:8000/doc/1016363361/c/info", host: "192.168.1.12", referrer: "http://192.168.1.12/7.0.0-132/web-apps/apps/spreadsheeteditor/main/index_loader.html?_dc=7.0.0-132&lang=en&frameEditorId=iframeEditor&parentOrigin=http://192.168.1.41"
2022/01/20 13:41:34 [error] 3990391#3990391: *6 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.40, server: , request: "GET /7.0.0-132/doc/1016363361/c/info?t=1642686094445 HTTP/1.1", upstream: "http://127.0.0.1:8000/doc/1016363361/c/info", host: "192.168.1.12", referrer: "http://192.168.1.12/7.0.0-132/web-apps/apps/spreadsheeteditor/main/index_loader.html?_dc=7.0.0-132&lang=en&frameEditorId=iframeEditor&parentOrigin=http://192.168.1.41"
2022/01/20 13:41:36 [error] 3990391#3990391: *6 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.40, server: , request: "GET /7.0.0-132/doc/1016363361/c/info?t=1642686096461 HTTP/1.1", upstream: "http://127.0.0.1:8000/doc/1016363361/c/info", host: "192.168.1.12", referrer: "http://192.168.1.12/7.0.0-132/web-apps/apps/spreadsheeteditor/main/index_loader.html?_dc=7.0.0-132&lang=en&frameEditorId=iframeEditor&parentOrigin=http://192.168.1.41"
2022/01/20 13:41:38 [error] 3990391#3990391: *6 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.40, server: , request: "GET /7.0.0-132/doc/1016363361/c/info?t=1642686098475 HTTP/1.1", upstream: "http://127.0.0.1:8000/doc/1016363361/c/info", host: "192.168.1.12", referrer: "http://192.168.1.12/7.0.0-132/web-apps/apps/spreadsheeteditor/main/index_loader.html?_dc=7.0.0-132&lang=en&frameEditorId=iframeEditor&parentOrigin=http://192.168.1.41"
2022/01/20 13:41:40 [error] 3990391#3990391: *6 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.40, server: , request: "GET /7.0.0-132/doc/1016363361/c/info?t=1642686100492 HTTP/1.1", upstream: "http://127.0.0.1:8000/doc/1016363361/c/info", host: "192.168.1.12", referrer: "http://192.168.1.12/7.0.0-132/web-apps/apps/spreadsheeteditor/main/index_loader.html?_dc=7.0.0-132&lang=en&frameEditorId=iframeEditor&parentOrigin=http://192.168.1.41"
2022/01/20 13:41:42 [error] 3990391#3990391: *6 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.40, server: , request: "GET /7.0.0-132/doc/1016363361/c/info?t=1642686102508 HTTP/1.1", upstream: "http://127.0.0.1:8000/doc/1016363361/c/info", host: "192.168.1.12", referrer: "http://192.168.1.12/7.0.0-132/web-apps/apps/spreadsheeteditor/main/index_loader.html?_dc=7.0.0-132&lang=en&frameEditorId=iframeEditor&parentOrigin=http://192.168.1.41"
2022/01/20 13:41:44 [error] 3990391#3990391: *6 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.40, server: , request: "GET /7.0.0-132/doc/1016363361/c/info?t=1642686104522 HTTP/1.1", upstream: "http://127.0.0.1:8000/doc/1016363361/c/info", host: "192.168.1.12", referrer: "http://192.168.1.12/7.0.0-132/web-apps/apps/spreadsheeteditor/main/index_loader.html?_dc=7.0.0-132&lang=en&frameEditorId=iframeEditor&parentOrigin=http://192.168.1.41"
2022/01/20 13:41:46 [error] 3990391#3990391: *6 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.40, server: , request: "GET /7.0.0-132/doc/1016363361/c/info?t=1642686106537 HTTP/1.1", upstream: "http://127.0.0.1:8000/doc/1016363361/c/info", host: "192.168.1.12", referrer: "http://192.168.1.12/7.0.0-132/web-apps/apps/spreadsheeteditor/main/index_loader.html?_dc=7.0.0-132&lang=en&frameEditorId=iframeEditor&parentOrigin=http://192.168.1.41"
2022/01/20 13:41:48 [error] 3990391#3990391: *6 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.40, server: , request: "GET /7.0.0-132/doc/1016363361/c/info?t=1642686108552 HTTP/1.1", upstream: "http://127.0.0.1:8000/doc/1016363361/c/info", host: "192.168.1.12", referrer: "http://192.168.1.12/7.0.0-132/web-apps/apps/spreadsheeteditor/main/index_loader.html?_dc=7.0.0-132&lang=en&frameEditorId=iframeEditor&parentOrigin=http://192.168.1.41"
2022/01/20 13:41:50 [error] 3990391#3990391: *6 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.40, server: , request: "GET /7.0.0-132/doc/1016363361/c/info?t=1642686110570 HTTP/1.1", upstream: "http://127.0.0.1:8000/doc/1016363361/c/info", host: "192.168.1.12", referrer: "http://192.168.1.12/7.0.0-132/web-apps/apps/spreadsheeteditor/main/index_loader.html?_dc=7.0.0-132&lang=en&frameEditorId=iframeEditor&parentOrigin=http://192.168.1.41"
2022/01/20 13:41:52 [error] 3990391#3990391: *6 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.40, server: , request: "GET /7.0.0-132/doc/1016363361/c/info?t=1642686112587 HTTP/1.1", upstream: "http://127.0.0.1:8000/doc/1016363361/c/info", host: "192.168.1.12", referrer: "http://192.168.1.12/7.0.0-132/web-apps/apps/spreadsheeteditor/main/index_loader.html?_dc=7.0.0-132&lang=en&frameEditorId=iframeEditor&parentOrigin=http://192.168.1.41"
2022/01/20 13:41:54 [error] 3990391#3990391: *6 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.40, server: , request: "GET /7.0.0-132/doc/1016363361/c/info?t=1642686114599 HTTP/1.1", upstream: "http://127.0.0.1:8000/doc/1016363361/c/info", host: "192.168.1.12", referrer: "http://192.168.1.12/7.0.0-132/web-apps/apps/spreadsheeteditor/main/index_loader.html?_dc=7.0.0-132&lang=en&frameEditorId=iframeEditor&parentOrigin=http://192.168.1.41"
2022/01/20 13:41:56 [error] 3990391#3990391: *6 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.40, server: , request: "GET /7.0.0-132/doc/1016363361/c/info?t=1642686116615 HTTP/1.1", upstream: "http://127.0.0.1:8000/doc/1016363361/c/info", host: "192.168.1.12", referrer: "http://192.168.1.12/7.0.0-132/web-apps/apps/spreadsheeteditor/main/index_loader.html?_dc=7.0.0-132&lang=en&frameEditorId=iframeEditor&parentOrigin=http://192.168.1.41"
2022/01/20 13:41:58 [error] 3990391#3990391: *6 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.40, server: , request: "GET /7.0.0-132/doc/1016363361/c/info?t=1642686118630 HTTP/1.1", upstream: "http://127.0.0.1:8000/doc/1016363361/c/info", host: "192.168.1.12", referrer: "http://192.168.1.12/7.0.0-132/web-apps/apps/spreadsheeteditor/main/index_loader.html?_dc=7.0.0-132&lang=en&frameEditorId=iframeEditor&parentOrigin=http://192.168.1.41"
2022/01/20 13:42:00 [error] 3990391#3990391: *6 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.40, server: , request: "GET /7.0.0-132/doc/1016363361/c/info?t=1642686120647 HTTP/1.1", upstream: "http://127.0.0.1:8000/doc/1016363361/c/info", host: "192.168.1.12", referrer: "http://192.168.1.12/7.0.0-132/web-apps/apps/spreadsheeteditor/main/index_loader.html?_dc=7.0.0-132&lang=en&frameEditorId=iframeEditor&parentOrigin=http://192.168.1.41"
2022/01/20 13:42:02 [error] 3990391#3990391: *6 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.40, server: , request: "GET /7.0.0-132/doc/1016363361/c/info?t=1642686122662 HTTP/1.1", upstream: "http://127.0.0.1:8000/doc/1016363361/c/info", host: "192.168.1.12", referrer: "http://192.168.1.12/7.0.0-132/web-apps/apps/spreadsheeteditor/main/index_loader.html?_dc=7.0.0-132&lang=en&frameEditorId=iframeEditor&parentOrigin=http://192.168.1.41"
2022/01/20 13:42:04 [error] 3990391#3990391: *6 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.40, server: , request: "GET /7.0.0-132/doc/1016363361/c/info?t=1642686124676 HTTP/1.1", upstream: "http://127.0.0.1:8000/doc/1016363361/c/info", host: "192.168.1.12", referrer: "http://192.168.1.12/7.0.0-132/web-apps/apps/spreadsheeteditor/main/index_loader.html?_dc=7.0.0-132&lang=en&frameEditorId=iframeEditor&parentOrigin=http://192.168.1.41"
2022/01/20 13:42:06 [error] 3990391#3990391: *6 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.40, server: , request: "GET /7.0.0-132/doc/1016363361/c/info?t=1642686126691 HTTP/1.1", upstream: "http://127.0.0.1:8000/doc/1016363361/c/info", host: "192.168.1.12", referrer: "http://192.168.1.12/7.0.0-132/web-apps/apps/spreadsheeteditor/main/index_loader.html?_dc=7.0.0-132&lang=en&frameEditorId=iframeEditor&parentOrigin=http://192.168.1.41"
2022/01/20 13:42:08 [error] 3990391#3990391: *6 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.40, server: , request: "GET /7.0.0-132/doc/1016363361/c/info?t=1642686128712 HTTP/1.1", upstream: "http://127.0.0.1:8000/doc/1016363361/c/info", host: "192.168.1.12", referrer: "http://192.168.1.12/7.0.0-132/web-apps/apps/spreadsheeteditor/main/index_loader.html?_dc=7.0.0-132&lang=en&frameEditorId=iframeEditor&parentOrigin=http://192.168.1.41"
2022/01/20 13:46:30 [error] 343#343: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.40, server: , request: "GET /7.0.0-132/doc/1016363361/c/info?t=1642686388300 HTTP/1.1", upstream: "http://127.0.0.1:8000/doc/1016363361/c/info", host: "192.168.1.12", referrer: "http://192.168.1.12/7.0.0-132/web-apps/apps/spreadsheeteditor/main/index_loader.html?_dc=7.0.0-132&lang=en&frameEditorId=iframeEditor&parentOrigin=http://192.168.1.41"
2022/01/20 13:46:32 [error] 343#343: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.40, server: , request: "GET /7.0.0-132/doc/1016363361/c/info?t=1642686392907 HTTP/1.1", upstream: "http://127.0.0.1:8000/doc/1016363361/c/info", host: "192.168.1.12", referrer: "http://192.168.1.12/7.0.0-132/web-apps/apps/spreadsheeteditor/main/index_loader.html?_dc=7.0.0-132&lang=en&frameEditorId=iframeEditor&parentOrigin=http://192.168.1.41"
2022/01/20 13:46:34 [error] 343#343: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.40, server: , request: "GET /7.0.0-132/doc/1016363361/c/info?t=1642686394922 HTTP/1.1", upstream: "http://127.0.0.1:8000/doc/1016363361/c/info", host: "192.168.1.12", referrer: "http://192.168.1.12/7.0.0-132/web-apps/apps/spreadsheeteditor/main/index_loader.html?_dc=7.0.0-132&lang=en&frameEditorId=iframeEditor&parentOrigin=http://192.168.1.41"
2022/01/20 13:46:36 [error] 343#343: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.40, server: , request: "GET /7.0.0-132/doc/1016363361/c/info?t=1642686396936 HTTP/1.1", upstream: "http://127.0.0.1:8000/doc/1016363361/c/info", host: "192.168.1.12", referrer: "http://192.168.1.12/7.0.0-132/web-apps/apps/spreadsheeteditor/main/index_loader.html?_dc=7.0.0-132&lang=en&frameEditorId=iframeEditor&parentOrigin=http://192.168.1.41"
2022/01/20 13:48:35 [error] 343#343: *6 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.41, server: , request: "GET /healthcheck HTTP/1.1", upstream: "http://127.0.0.1:8000/healthcheck", host: "192.168.1.12"
2022/01/20 13:48:47 [error] 343#343: *8 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.41, server: , request: "GET /healthcheck HTTP/1.1", upstream: "http://127.0.0.1:8000/healthcheck", host: "192.168.1.12"
2022/01/20 13:50:27 [error] 343#343: *12 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.41, server: , request: "GET /healthcheck HTTP/1.1", upstream: "http://127.0.0.1:8000/healthcheck", host: "192.168.1.12"
2022/01/20 13:50:37 [error] 343#343: *14 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.40, server: , request: "GET /healthcheck HTTP/1.1", upstream: "http://127.0.0.1:8000/healthcheck", host: "192.168.1.12"
2022/01/20 13:50:37 [error] 343#343: *14 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.40, server: , request: "GET /favicon.ico HTTP/1.1", upstream: "http://127.0.0.1:8000/favicon.ico", host: "192.168.1.12", referrer: "http://192.168.1.12/healthcheck"
2022/01/20 13:51:48 [error] 343#343: *17 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.41, server: , request: "GET /healthcheck HTTP/1.1", upstream: "http://127.0.0.1:8000/healthcheck", host: "192.168.1.12"
2022/01/20 13:51:56 [error] 343#343: *19 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.41, server: , request: "GET /healthcheck HTTP/1.1", upstream: "http://127.0.0.1:8000/healthcheck", host: "192.168.1.12"
2022/01/20 14:08:27 [error] 353#353: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.41, server: , request: "GET /healthcheck HTTP/1.1", upstream: "http://127.0.0.1:8000/healthcheck", host: "192.168.1.12"
2022/01/20 14:17:33 [error] 353#353: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.41, server: , request: "GET /healthcheck HTTP/1.1", upstream: "http://127.0.0.1:8000/healthcheck", host: "192.168.1.12"
2022/01/20 14:18:03 [error] 353#353: *5 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.41, server: , request: "GET /healthcheck HTTP/1.1", upstream: "http://127.0.0.1:8000/healthcheck", host: "192.168.1.12"
2022/01/20 14:18:14 [error] 353#353: *7 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.1.41, server: , request: "GET /healthcheck HTTP/1.1", upstream: "http://127.0.0.1:8000/healthcheck", host: "192.168.1.12"
DOC Service Log:
<--- Last few GCs --->
[61738:0x36a9300] 13961932 ms: Mark-sweep 4064.7 (4134.5) -> 4051.6 (4137.2) MB, 6302.0 / 0.6 ms (average mu = 0.126, current mu = 0.033) allocation failure scavenge might not succeed
[61738:0x36a9300] 13966901 ms: Mark-sweep 4067.7 (4137.2) -> 4054.5 (4140.2) MB, 4825.6 / 0.6 ms (average mu = 0.080, current mu = 0.029) allocation failure scavenge might not succeed
<--- JS stacktrace --->
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
1: 0x8c60e0 node::Abort() [/var/www/onlyoffice/documentserver/server/DocService/docservice]
2: 0x8221d1 node::FatalError(char const*, char const*) [/var/www/onlyoffice/documentserver/server/DocService/docservice]
3: 0x9e2af6 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/var/www/onlyoffice/documentserver/server/DocService/docservice]
4: 0x9e2d8d v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/var/www/onlyoffice/documentserver/server/DocService/docservice]
5: 0xb845c5 [/var/www/onlyoffice/documentserver/server/DocService/docservice]
6: 0xb85017 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [/var/www/onlyoffice/documentserver/server/DocService/docservice]
7: 0xb8db98 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/var/www/onlyoffice/documentserver/server/DocService/docservice]
8: 0xb911c0 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/var/www/onlyoffice/documentserver/server/DocService/docservice]
9: 0xb5a862 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/var/www/onlyoffice/documentserver/server/DocService/docservice]
10: 0xe6148e v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/var/www/onlyoffice/documentserver/server/DocService/docservice]
11: 0x17585f9 [/var/www/onlyoffice/documentserver/server/DocService/docservice]