@apollon77:
Das was passiert ist sehr einfach: Irgendein Prozess frisst speicher, der Server rennt ins Swap und das erzeugt I/O und damit wird alles lllaaanggggsssaaammmm [emoji3]
Irgendwann schlägt der OOM-Killer vom Betriebssystem zu und killt idealerweise den verursachenden prozess … das ist das wo es sich plötzlich wieder erholt.
Und da du das beschreibst solltest Du in /var/log/syslog in jedem Fall infos finden weil der oom-killer da hinschreibt was er warum abgeschossen hat.
Und ja du musst den/die Prozesse finden der hier Speicher fressen.
Wenn Du kannst (und es regelmäßig genug passiert) lass einfach ne Shell offen und dort "top" drin laufen ... dann siehst Du wenn es passiert mit höherer Wahrscheinlichkeit was los ist.
Zu htop: Man muss an sich nix installieren ... "top" ist als Standardbefehl immer da bei Linux [emoji3] und liefert für den Fall sinnvollere Informationen `
Ich habe folgendes im syslog gefunden.
In diesen fall killt er io.web. Das gleiche passiert auch mit io.javascript hm.rega0 und io.history
MOD-Edit by eric2905 - Log in Spoiler- und Code-Tags gesetzt
! ````
Oct 19 09:19:24 zotac kernel: [1207257.720918] Out of memory: Kill process 25749 (iobroker.js-con) score 11 or sacrifice child
Oct 19 09:19:24 zotac kernel: [1207257.722445] Killed process 25814 (io.web.0) total-vm:1233264kB, anon-rss:35888kB, file-rss:1648kB
Oct 19 09:20:32 zotac kernel: [1207323.947924] iobroker.js-con invoked oom-killer: gfp_mask=0x26000c0, order=2, oom_score_adj=0
Oct 19 09:20:32 zotac kernel: [1207323.947932] iobroker.js-con cpuset=/ mems_allowed=0
Oct 19 09:20:32 zotac kernel: [1207323.947942] CPU: 3 PID: 25749 Comm: iobroker.js-con Tainted: G OE 4.4.0-96-generic #119-Ubuntu
Oct 19 09:20:32 zotac kernel: [1207323.947945] Hardware name: Motherboard by ZOTAC ZBOX-CI320NANO series/ZBOX-CI320NANO series, BIOS B219P028 04/27/2016
Oct 19 09:20:32 zotac kernel: [1207323.947948] 0000000000000286 687c7aef17eaa985 ffff880114c8bb10 ffffffff813fabd3
Oct 19 09:20:32 zotac kernel: [1207323.947953] ffff880114c8bcc8 ffff880016a36600 ffff880114c8bb80 ffffffff8120c45e
Oct 19 09:20:32 zotac kernel: [1207323.947958] ffff8800ae04cfd0 ffff8800ae04cc80 ffff880114c8bcc8 ffff880114c8bbd8
Oct 19 09:20:32 zotac kernel: [1207323.947962] Call Trace:
Oct 19 09:20:32 zotac kernel: [1207323.947973] [<ffffffff813fabd3>] dump_stack+0x63/0x90
Oct 19 09:20:32 zotac kernel: [1207323.947980] [<ffffffff8120c45e>] dump_header+0x5a/0x1c5
Oct 19 09:20:32 zotac kernel: [1207323.947985] [<ffffffff81192e42>] oom_kill_process+0x202/0x3c0
Oct 19 09:20:32 zotac kernel: [1207323.947989] [<ffffffff81193269>] out_of_memory+0x219/0x460
Oct 19 09:20:32 zotac kernel: [1207323.947994] [<ffffffff81199248>] __alloc_pages_slowpath.constprop.88+0x938/0xad0
Oct 19 09:20:32 zotac kernel: [1207323.947999] [<ffffffff81199666>] __alloc_pages_nodemask+0x286/0x2a0
Oct 19 09:20:32 zotac kernel: [1207323.948004] [<ffffffff8119971b>] alloc_kmem_pages_node+0x4b/0xc0
Oct 19 09:20:32 zotac kernel: [1207323.948009] [<ffffffff8107eaae>] copy_process+0x1be/0x1b30
Oct 19 09:20:32 zotac kernel: [1207323.948014] [<ffffffff811c2677>] ? handle_mm_fault+0x1277/0x1820
Oct 19 09:20:32 zotac kernel: [1207323.948019] [<ffffffff8134a753>] ? security_file_alloc+0x33/0x50
Oct 19 09:20:32 zotac kernel: [1207323.948023] [<ffffffff810805b0>] _do_fork+0x80/0x360
Oct 19 09:20:32 zotac kernel: [1207323.948028] [<ffffffff81080939>] SyS_clone+0x19/0x20
Oct 19 09:20:32 zotac kernel: [1207323.948033] [<ffffffff81843272>] entry_SYSCALL_64_fastpath+0x16/0x71
Oct 19 09:20:32 zotac kernel: [1207323.948037] Mem-Info:
Oct 19 09:20:32 zotac kernel: [1207323.948046] active_anon:616112 inactive_anon:230584 isolated_anon:0
Oct 19 09:20:32 zotac kernel: [1207323.948046] active_file:1272 inactive_file:1076 isolated_file:179
Oct 19 09:20:32 zotac kernel: [1207323.948046] unevictable:915 dirty:0 writeback:0 unstable:0
Oct 19 09:20:32 zotac kernel: [1207323.948046] slab_reclaimable:10669 slab_unreclaimable:19334
Oct 19 09:20:32 zotac kernel: [1207323.948046] mapped:2996 shmem:2489 pagetables:32255 bounce:0
Oct 19 09:20:32 zotac kernel: [1207323.948046] free:21689 free_pcp:0 free_cma:0
Oct 19 09:20:32 zotac kernel: [1207323.948053] Node 0 DMA free:15728kB min:272kB low:340kB high:408kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15984kB managed:15900kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:24kB slab_unreclaimable:100kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Oct 19 09:20:32 zotac kernel: [1207323.948063] lowmem_reserve[]: 0 2829 3788 3788 3788
Oct 19 09:20:32 zotac kernel: [1207323.948069] Node 0 DMA32 free:54056kB min:50272kB low:62840kB high:75408kB active_anon:2069752kB inactive_anon:526900kB active_file:3084kB inactive_file:2616kB unevictable:3320kB isolated(anon):0kB isolated(file):484kB present:3018248kB managed:2937556kB mlocked:3320kB dirty:0kB writeback:0kB mapped:8164kB shmem:7528kB slab_reclaimable:28360kB slab_unreclaimable:46064kB kernel_stack:50016kB pagetables:76072kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:68 all_unreclaimable? no
Oct 19 09:20:32 zotac kernel: [1207323.948078] lowmem_reserve[]: 0 0 958 958 958
Oct 19 09:20:32 zotac kernel: [1207323.948083] Node 0 Normal free:16972kB min:17032kB low:21288kB high:25548kB active_anon:394696kB inactive_anon:395436kB active_file:2004kB inactive_file:1688kB unevictable:340kB isolated(anon):0kB isolated(file):232kB present:1048576kB managed:981712kB mlocked:340kB dirty:0kB writeback:0kB mapped:3820kB shmem:2428kB slab_reclaimable:14292kB slab_unreclaimable:31172kB kernel_stack:16448kB pagetables:52948kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:244 all_unreclaimable? no
Oct 19 09:20:32 zotac kernel: [1207323.948091] lowmem_reserve[]: 0 0 0 0 0
Oct 19 09:20:32 zotac kernel: [1207323.948096] Node 0 DMA: 24kB (H) 38kB (EH) 316kB (EH) 132kB (H) 264kB (EH) 1128kB (E) 2256kB (EH) 1512kB (E) 21024kB (EH) 22048kB (ME) 24096kB (M) = 15728kB
Oct 19 09:20:32 zotac kernel: [1207323.948117] Node 0 DMA32: 135654kB (UME) 108kB (M) 016kB 032kB 064kB 0128kB 0256kB 0512kB 01024kB 02048kB 04096kB = 54340kB
Oct 19 09:20:32 zotac kernel: [1207323.948131] Node 0 Normal: 42594kB (UM) 58kB (M) 016kB 032kB 064kB 0128kB 0256kB 0512kB 01024kB 02048kB 0*4096kB = 17076kB
Oct 19 09:20:32 zotac kernel: [1207323.948146] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Oct 19 09:20:32 zotac kernel: [1207323.948148] 6098 total pagecache pages
Oct 19 09:20:32 zotac kernel: [1207323.948151] 536 pages in swap cache
Oct 19 09:20:32 zotac kernel: [1207323.948154] Swap cache stats: add 1781081, delete 1780545, find 1513889/1715381
Oct 19 09:20:32 zotac kernel: [1207323.948156] Free swap = 0kB
Oct 19 09:20:32 zotac kernel: [1207323.948158] Total swap = 1983484kB
Oct 19 09:20:32 zotac kernel: [1207323.948160] 1020702 pages RAM
Oct 19 09:20:32 zotac kernel: [1207323.948162] 0 pages HighMem/MovableOnly
Oct 19 09:20:32 zotac kernel: [1207323.948164] 36910 pages reserved
Oct 19 09:20:32 zotac kernel: [1207323.948165] 0 pages cma reserved
Oct 19 09:20:32 zotac kernel: [1207323.948167] 0 pages hwpoisoned
Oct 19 09:20:32 zotac kernel: [1207323.948169] [ pid ] uid tgid total_vm rss nr_ptes nr_pmds swapents oom_score_adj name
Oct 19 09:20:32 zotac kernel: [1207323.948188] [ 356] 0 356 10151 1047 22 3 554 0 systemd-journal
Oct 19 09:20:32 zotac kernel: [1207323.948195] [ 395] 0 395 25742 35 17 3 564 0 lvmetad
Oct 19 09:20:32 zotac kernel: [1207323.948201] [ 419] 0 419 11200 340 23 3 278 -1000 systemd-udevd
Oct 19 09:20:32 zotac kernel: [1207323.948207] [ 695] 0 695 8563 349 22 3 5606 0 mount.ntfs-3g
Oct 19 09:20:32 zotac kernel: [1207323.948212] [ 761] 100 761 25596 292 20 3 53 0 systemd-timesyn
Oct 19 09:20:32 zotac kernel: [1207323.948217] [ 851] 0 851 77160 261 19 4 561 0 lxcfs
Oct 19 09:20:32 zotac kernel: [1207323.948222] [ 856] 0 856 7158 306 18 3 41 0 systemd-logind
Oct 19 09:20:32 zotac kernel: [1207323.948227] [ 860] 104 860 64099 538 26 3 96 0 rsyslogd
Oct 19 09:20:32 zotac kernel: [1207323.948232] [ 862] 107 862 10726 216 25 4 62 -900 dbus-daemon</ffffffff81843272></ffffffff81080939></ffffffff810805b0></ffffffff8134a753></ffffffff811c2677></ffffffff8107eaae></ffffffff8119971b></ffffffff81199666></ffffffff81199248></ffffffff81193269></ffffffff81192e42></ffffffff8120c45e></ffffffff813fabd3>