How to lower disk writes (for extending life of an sd card)

[Tomasz Nowak]

Hi,
I'm new to ioBroker but I have been using FHEM, deCONZ, Node-RED, etc. before.
I have successfully installed ioBroker in Docker (using Portainer) on yet another Raspberry Pi 3B+ in my house.
It runs on Sandisk Industrial microsd card and and I'd like to to stay that way.
I checked system disk writes with iotop and found out that iobroker.js-controller is writing relatively much to a files:
/opt/iobroker/iobroker-data/states.json
/opt/iobroker/iobroker-data/states.json.bak

I understand, that this is a permanent json storage of in-memory states database and is written probably every 30 seconds.
Is this (frequent interval) required for anything?
My RPi3 won't be powered off so keeping all in memory is fine and if reboot happens I don't care about states or its history - they would be just read from devices(es) again.

So my question is - how do I extend write interval to states.json to 8 or 24 hours?
I didn't find such setting in iobroker.json

[paul53]

@Tomasz-Nowak sagte:

how do I extend write interval to states.json to 8 or 24 hours?

In file node_modules/iobroker.js-controller/lib/states/statesInMemFileDB.js search for 30000 and change it to a higher value (7 times). Then iobroker restart.

[Tomasz Nowak]

@paul53
Thanks, works like a charm. I've put 28800000 for 8-hour save interval and instantly sd card writes dropped to almost zero on that PRI. Would be great if that interval could be configurable somewhere in the future

[paul53]

@Tomasz-Nowak sagt:

Would be great if that interval could be configurable somewhere in the future

Please create an issue on Github.

[apollon77]

@Tomasz-Nowak sagte in How to lower disk writes (for extending life of an sd card):

My RPi3 won't be powered off so keeping all in memory is fine and if reboot happens I don't care about states or its history - they would be just read from devices(es) again.

I don't know if it is really that easy as soon as you have more complex scripts with "Non device states". If you choose that way then you need to make sure that all your scripts can work with "missing" "outdated" state values ... this will be likely hard when you do not only use device values, but also logical values.
So when you increase it that much - know what you are doing and which effects it has!!

[paul53]

@apollon77 sagte:

this will be likely hard when you do not only use device values, but also logical values.

A reboot / killing of ioBroker immediately writes the states from RAM to the file. Only in case of a system crash (power failure) such a long time is problematic.

[apollon77]

@paul53 ... or a process crash (yes there is come code that tries to write it also when crashing) ... but (e.g. a SIGTERM because of OOM might also not write values).

It is all about "knowing what one is doing"

[Tomasz Nowak]

@paul53 , @apollon77
Thanks for your hints, guys. For now I'm just handling Ecovacs Deebot vacuuming robot (and Daikin aircon soon, probably) over MQTT with no logic within ioBroker. So I hope I don't risk any house disaster with that setting

@paul53, I've created an issue, thanks.

[Dutchman]

just as suggestion, to avoid load read/write loads on storage and move the to Memory isn't Redis providing more efficient state handling here ?

[Tomasz Nowak]

I thought about it but AFAIK Redis also needs to write to disk / sd card. I have relatively small number of states so I believe creating yet another docker instance for Redis would be kind of overkill in my case.

[Mic]

@Tomasz-Nowak
I am using Redis successfully for several months. I have installed it on the same system (Debian) on which ioBroker is installed - but no Raspberry experience (I am using a Zotac barebone with Proxmox), though.
See: https://github.com/ioBroker/ioBroker.js-controller#state-and-objects-databases-and-files
Also, see: https://redis.io/topics/persistence
You may want to look into it again, but not sure how the performance is gonna be on a Raspberry.

[Tomasz Nowak]

@Mic
I've read that before. They say good for many objects such has 10000 for performance reasons. I have like 50 objects
And I don't want make it more complex (you don't install another services in docker instance, you need another docker)

[Dutchman]

guys wait ...., don't ix performance and io topics please

we are discussing write access to storage, Redis works in memory so less write on disk/card was my hypothese

[paul53]

@Dutchman sagte:

Redis works in memory so less write on disk/card

Per default Redis writes every 5 minutes to disk/card.

[Dutchman]

@paul53 sagte in How to lower disk writes (for extending life of an sd card):

@Dutchman sagte:

Redis works in memory so less write on disk/card

Per default Redis writes every 5 minutes to disk/card.

Better than every 30 seconds

[Mic]

@Dutchman sagte in How to lower disk writes (for extending life of an sd card):

Better than every 30 seconds

Exactly, and the default value can certainly be increased to whatever is needed. I would give Redis a try.
Also,
@Tomasz-Nowak sagte in How to lower disk writes (for extending life of an sd card):

They say good for many objects such has 10000 for performance reasons

I would assume this statement in the documentation was just made for the ease of use of ioBroker in general: Without like Redis, typically ioBroker's performance is good if you have way less than 10000+ objects. You are having a different use case (limited disc r/w per period). Not sure how the performance (like CPU and memory usage) of Redis is on a Raspberry, but I guess pretty low.