- Home
- Deutsch
- Skripten / Logik
- JavaScript
- [Vorlage] JS: Log-Datei aufbereiten für VIS
[Vorlage] JS: Log-Datei aufbereiten für VIS
This topic has been deleted. Only users with topic management privileges can see it.
-
@Glasfaser
@Negalein
Vielen Dank für euer Feedback
@Glasfaser sagte in [Vorlage] JS: Log-Datei aufbereiten für VIS:
Ich würde das Skript noch auf Github legen
Guter Hinweis, erledigt (da ist auch das Filterbeispiel drin): https://github.com/Mic-M/iobroker.logfile-script/blob/master/alexa/alexa-log-script.js
-
Hm, ist das jetzt ein eigenes Skript?
-
@sigi234 sagte in [Vorlage] JS: Log-Datei aufbereiten für VIS:
Hm, ist das jetzt ein eigenes Skript?
das kurze Script wird benötigt, um mit den neuen Log-Script die Alexa-History auszulesen.
-
@Negalein sagte in [Vorlage] JS: Log-Datei aufbereiten für VIS:
@Mic
Ich bekomm folgenden Error und das Log-Script geht auf Pause (Gelb).Leider stellt das Forum hier Regex in Codes nicht korrekt dar, bitte kopiere den Filter noch mal von hier:
https://github.com/Mic-M/iobroker.logfile-script/blob/master/alexa/alexa-log-script.js
(Zeile: "clean:" ersetzen reicht) -
-
@sigi234 sagte in [Vorlage] JS: Log-Datei aufbereiten für VIS:
Muss ich das was ändern und es in das Große Skript kopieren?
Nein, sieht gut aus. Ist ein separates Script nur für Alexa.
-
@sigi234 sagte in [Vorlage] JS: Log-Datei aufbereiten für VIS:
Muss ich das was ändern
Nein
und es in das Große Skript kopieren
Zeile 18 bis 30 ins Log-Script
-
-
@Negalein sagte in [Vorlage] JS: Log-Datei aufbereiten für VIS:
perfekt wie immer
Dem schließe ich mich an!
-
Dein Beispiel-Projekt würde ich auch noch anheften …
... sonst geht es auch hier unter
-
-
@Negalein sagte in [Vorlage] JS: Log-Datei aufbereiten für VIS:
perfekt wie immer
@sigi234 sagte in [Vorlage] JS: Log-Datei aufbereiten für VIS:
Dem schließe ich mich an!
Vielen Dank, auch an Sigi für den Alexa-Befehl
-
@Glasfaser sagte in [Vorlage] JS: Log-Datei aufbereiten für VIS:
Dein Beispiel-Projekt würde ich auch noch anheften …
... sonst geht es auch hier unterDanke, ist hier dokumentiert, und nun auch verlinkt: https://github.com/Mic-M/iobroker.logfile-script/blob/master/README.md#3-log-ausgaben-im-vis-darstellen
-
Ich überlege gerade wie ich am Besten ein aufklappbares Menü mache da es sicher noch mehr wird.
-
@sigi234 sagte in [Vorlage] JS: Log-Datei aufbereiten für VIS:
Ich überlege gerade wie ich am Besten ein aufklappbares Menü mache da es sicher noch mehr wird.
Vielleicht über List Widget ...
-
Ich habe das Skript gerade importiert, auf meine Umgebung angepasst und gestartet.
Ich erhalte folgedn Fehlermeldung:
javascript.0 2020-03-02 07:41:26.748 error (3135) at Script.runInContext (vm.js:133:20) javascript.0 2020-03-02 07:41:26.748 error (3135) at script.js.java.abfragen.log_vis_neu:540:1 javascript.0 2020-03-02 07:41:26.748 error (3135) at init (script.js.java.abfragen.log_vis_neu:544:5) javascript.0 2020-03-02 07:41:26.748 error (3135) ReferenceError: onLogUnregister is not defined javascript.0 2020-03-02 07:41:26.748 error (3135) ^ javascript.0 2020-03-02 07:41:26.748 error (3135) onLogUnregister(G_LogHandler); javascript.0 2020-03-02 07:41:26.748 error (3135) script.js.java.abfragen.log_vis_neu: script.js.java.abfragen.log_vis_neu:544 -
@Mic sagte in [Vorlage] JS: Log-Datei aufbereiten für VIS:
@ple sagte in [Vorlage] JS: Log-Datei aufbereiten für VIS:
Vielleicht hast du eine Idee, wie ich dein Script nutzen kann als Meldearchiv.
Eigentlich will ich mir in einer Table nur anzeigen lassen, wenn ein Aktor auf true geht, oder sonst was.
Ähnlich Fensterzähler oder dein Garagentor. Nur soll die Meldung auch wieder weggehen, wenn z.B. dein Garagentor nicht mehr klemmt.
Hast du da vielleicht eine Idee, wie man sowas umsetzen könnte?
Mein rudimentärer Anfang ist hier mal dokumentiert.
https://forum.iobroker.net/topic/30324/array-erstellen-mit-prüfung-für-json-tabelle/9Hi,
das Log-Script ist nicht für deinen Anwendungsfall gedacht, es nimmt ausschließlich die Logs als Quelle. Ebenso ist es ein "Log", daher werden ältere Werte nicht nachträglich angepasst.Das Script ist dahingehend auch super, da man die Logs auch durchsehen kann.
Ich frage mich gerade, wie man das Script umbauen könnte, dass die Log zwar wie jetzt geschrieben werden und dieses auch in einem Json gepackt wird, jedoch eine Bediungung eintritt, die das löschen auslöst auch wieder aus der Json entfernt.Vorstellen könnte ich mir das so, Fenster links ist offen, ich schreiben ein Log, Fenster wieder zu, ich überprüfe die Json, finde "Fenster links ist offen" und lösche das aus der Json. Das schreiben und Filtern dürfte ja mit deinem Script so funktionieren, nur das löschen wäre dahingehend interessant.
Vielleicht wäre das ein Extra Script, was die Json dahingehend nachträglich bearbeitet, nur leider reichen meine Kenntnisse dafür leider nicht aus.
Vielleicht hast du beim Kaffee eine ruhige Minute, wie man das vielleicht umsetzen könnte, den Kaffee gebe ich dir gerne aus
-
Hallo,
top Script besten Dank an @mic noch mal.
Habe das Problem das bei mir die Blacklist-Global nicht berücksichtigt wird.
-
@Icube
Das Komma hinter dem letzten Eintrag in der global Blacklist muss weg, sonst erwartet er noch einen Eintrag.Gleiches gilt für die Filter. Wenn die eckige Klammer zugeht, darf davor kein Komma stehen.
Hoffe das löst dein Problem.
-
@blackeagle998 habe das Komma entfernt hat aber nichts gebracht
anbei das script
/************************************************************************************************************************* * --------------------------- * Log Script für ioBroker zum Aufbereiten des Logs für Visualisierungen (vis), oder um * auf Log-Ereignisse zu reagieren. * --------------------------- * * Das Script nimmt jeden neuen Logeintrag des ioBrokers und wendet entsprechend gesetzte * Filter an, um den Eintrag dann in den entsprechenden Datenpunkten dieses Scripts abzulegen. * Es stehen auch JSON-Datenpunkte zur Verfügung, mit diesen kann im vis eine * Tabelle ausgegeben werden (z.B. über das Widget 'basic - Table' oder 'materialdesign - Table'). * * Aktuelle Version: https://github.com/Mic-M/iobroker.logfile-script * Support: https://forum.iobroker.net/topic/13971/vorlage-log-datei-aufbereiten-f%C3%BCr-vis-javascript * Autor: Mic (ioBroker) | Mic-M (github) * ----------------------------------------------------------------------------------------------------------------------- * VORAUSSETZUNGEN * 1.) Nur falls Datenpunkte unterhalb '0_userdata.0' abgelegt werden sollen: * In der Instanz des JavaScript-Adapters die Option [Erlaube das Kommando "setObject"] aktivieren. * Siehe auch: https://github.com/Mic-M/iobroker.createUserStates * 2.) Dieses Script benötigt die JavaScript-Adapter-Version 4.3.0 (2019-10-09) oder höher. * Wer eine ältere Version einsetzt: Bitte Script-Version 2.0.2 verwenden. * ===================================================================================== * ----------------------------------------------------------------------------------------------------------------------- * Change Log: * 4.10 Mic + JSON: add support for individual items in column provided through log. See * 4.9 Mic + Add "Heute"/"Gestern" (today/yesterday) option for dates in JSON output instead of actual date * - Fix: jsonDateFormat: correction of month/minute. Month/Day/Year (mm/dd/yy/yyyy) now require lower case * letters, Hour/Minute/Second (HH/MM/SS) upper case letters to differenciate month (mm) vs. minute (MM) * 4.8 Mic + Allow regular expressions in BLACKLIST_GLOBAL * + Allow regular expressions in LOG_FILTER: 'blacklist', 'clean', 'filter_all', 'filter_any' * - Fix: Log line regex, which did not allow upper case letters in host (source) * 4.7 Mic - Fixed issue if NUMBER_OF_VIS_VIEWS = 0 * 4.6 Mic - Pushing 'Log-Script.visView1.clearJSON' will update timestamp of 'Log-Script.logXXX.clearJSON'. * See: https://forum.iobroker.net/post/375932 * 4.5.0 Mic * Simply moved 4.5Alpha into 4.5.0 due to successful user tests. No code changes since 4.5Alpha. * 4.5Alpha Mic + Add new states '.All.visViewX' to switch between JSON tables in VIS. * + Add new state '.All.clearAllJSON' to clear all JSON logs of all defined filters at once. * + Add new state '.All.kastTimeUpdated' to provide the time of the last update (schedule) * of the script in a state for Visualization. * 4.0.1 Mic - Add "jsonDateFormat: 'dd.mm. hh:mm'," to FILTER_LOG, id 'all'. * 4.0 Mic + To allow individual settings per each defined LOG_FILTER, the following global * settings were moved to LOG_FILTER: * * JSON_DATE_FORMAT -> jsonDateFormat * * JSON_LEN -> jsonLogLength * * JSON_NO_ENTRIES -> jsonMaxLines * * JSON_APPLY_CSS_LIMITED_TO_LEVEL -> jsonCssToLevel * * L_SORT_ORDER_DESC -> sortDescending * + Code improvements * + Renamed LOG_NO_OF_ENTRIES to MAX_LOG_LINES * ---------------------------------------------------------------------------------------------------- * 3.4 Mic + Support both '0_userdata.0' and 'javascript.x' for state creation * 3.3 Mic - Fix state path * 3.2 Mic + Create all states under 0_userdata.0, and no longer under javascript.<instance> (like javascript.0) * 3.1 Mic + Change to stable as tests were successful * + Add new option REMOVE_PID: The js-controller version 2.0+ adds the PID number inside brackets * to the beginning of the message. Setting REMOVE_PID = false will remove it. * 3.0Alpha Mic + Major Change: JavaScript adapter 4.3+ now provides onLog() function: * https://github.com/ioBroker/ioBroker.javascript/blob/master/docs/en/javascript.md#onlog * We are using this new function to streamline this log script tremendously and to remove node-tail. * ---------------------------------------------------------------------------------------------------- * 2.0.2 Mic + Changed certain functions to async to get rid of setTimout() and for the sake of better error handling. * + startTailingProcess(): ensure the tailing starts if the file is present (wait to be created) * 2.0.1a Mic Removed constant MERGE_LOGLINES_ACTIVE * 2.0.0a Mic Major improvements and fixes: * + Change from instant state update to schedule (STATE_UPDATE_SCHEDULE). The instant update, so once * new log entries coming in, caused several issues (setting and getting state values (getState() and * setState()) within <1ms simply does not work. * - Fix issue with merging log lines * + Moved global option MERGE_LOGLINES_ACTIVE to LOG_FILTER, for allowing turning on/off for each filter id. * + Several other code improvements * Note: For upgrading from previous version: replace script entirely, re-enter all your options, * and delete all existing states prior to first activation of this script. * ---------------------------------------------------------------------------------------------------- * 1.5.1 Mic - Set option MERGE_LOGLINES_ACTIVE to 'false' as default, as users reported issues. See * https://forum.iobroker.net/post/288772 . Also option MERGE_LOGLINES_ACTIVE being marked as "experimental" * in the comments. Requires further investigation. * 1.5 Mic - Fix issue with option MERGE_LOGLINES_ACTIVE * 1.4 Mic + New option MERGE_LOGLINES_TXT for an individual (e.g. localized) string other than 'entries'. - Fix JSON span class closing * 1.3 Mic + New option MERGE_LOGLINES_ACTIVE: Merge Loglines with same log message to only one line and adds leading * '[123 entries]' to log message. * 1.2 Mic - Fixed issue #6 (Button javascript.0.Log-Script.logXxxx.clearJSON not working reliably) * 1.1 Mic + 1. 1.0x script seems to work reliable per user feedback and my own test, so pushing into 1.1 stable. * + New state '.logMostRecent': provides just the most recent log entry to work with "on / * subscribe" on this state and trigger actions accordingly. * 1.02 alpha Mic - fix restarting at 0:00 (note: restarting is needed due to log file name change) * 1.01 alpha Mic - fix: creating new file system log file only if not yet existing * 1.00 alpha Mic + Entirely recoded to implement node-tail (https://github.com/lucagrulla/node-tail). * ---------------------------------------------------------------------------------------------------- * 0.8.1 Mic - Fix: L_SORT_ORDER_DESC was not defined (renamed constant name was not changed in config) * 0.8 Mic - Fix: Script caused a "file not found" error if executed right at or shortly after midnight. * 0.7 Mic - Fix: States "...clearDateTime" will not get an initial date value on first script start, * also fix for "on({id: ". * 0.6 Mic + Put 0.5.1 BETA into stable * + New option L_APPLY_CSS. If true, it will add <span class='log-info'>xxx</span> * to each log string. 'log-info' for level info, 'log-error' for error, etc. * This makes it easy to format a JSON table with CSS. * 0.5.1 BETA Mic + New States "Clear JSON log ..." and "Clear JSON log - Date/Time ...". * When the button "Clear JSON log" is pushed, the current date/time * will be set into the date/time state. Once refreshed * (per schedule in the script, e.g. after 2 minutes), the JSON * will be cleaned and display just newer logs. * Use Case: In vis, you can now add a button "clear log" or * "Mark as read". If you hit the button, the log will be * cleared and just new log items will be displayed. * *** THIS IS STILL BEING TESTED *** therefore a beta release... * 0.5 Mic + New parameter 'clean' to remove certain strings * from the log line. * + New parameter 'columns' for JSON output to specify which columns * to be shown, and in which order. * + New state "JSONcount" to have the number of log lines in state * - Fixed a few issues * 0.4 Mic - Bug fix: improved validation of log line consistency * 0.3 Mic + Added filtering and blacklist * - Several fixes * 0.2 Mic - Bug fix: corrected wrong function name * 0.1 Mic * Initial release ************************************************************************************************************************/ /******************************************************************************* * Konfiguration: Pfade ******************************************************************************/ // Pfad, unter dem die States (Datenpunkte) in den Objekten angelegt werden. // Es wird die Anlage sowohl unterhalb '0_userdata.0' als auch 'javascript.x' unterstützt. const LOG_STATE_PATH = '0_userdata.0.Log-Script'; // Pfad zum Log-Verzeichnis auf dem Server. // Standard-Pfad unter Linux: '/opt/iobroker/log/'. Wenn das bei dir auch so ist, dann einfach belassen. const LOG_FS_PATH = '/opt/iobroker/log/'; /******************************************************************************* * Konfiguration: Alle Logeinträge - Global ******************************************************************************/ // Zahl: Maximale Anzahl der letzten Logeinträge in den Datenpunkten. Alle älteren werden entfernt. // Bitte nicht allzu viele behalten, denn das kostet Performance. const MAX_LOG_LINES = 100; // Der js-Controller Version 2.0 oder größer fügt Logs teils vorne die PID in Klammern hinzu, // also z.B. "(12234) Terminated (15): Without reason". // Mit dieser Option lassen sich die PIDs aus den Logzeilen entfernen. const REMOVE_PID = true; /** * Schwarze Liste (Black list) * Falls einer dieser Satzteile/Begriffe in einer Logzeile enthalten ist, dann wird der Log-Eintrag * komplett ignoriert, egal was weiter unten eingestellt wird. * Dies dient dazu, um penetrante Logeinträge gar nicht erst zu berücksichtigen. * Bitte beachten: * 1. Es ist sowohl String als auch Regex erlaubt. Falls String: es wird auf teilweise Übereinstimmung geprüft. * 2. Bestehende Datenpunkt-Inhalte dieses Scripts bei Anpassung dieser Option werden nicht nachträglich neu gefiltert, * sondern nur alle neu hinzugefügten Log-Einträge ab Speichern des Scripts werden berücksichtigt. */ const BLACKLIST_GLOBAL = [ '<==Disconnect system.user.admin from ::ffff:', // web.0 Adapter /registered [0-9]+ subscriptions and [0-9]+ schedules/, // Beispiel für Regex /instance system\.adapter\.[^\s]* terminated with code 0 \((NO_ERROR|OK)\)/, // Weiteres Regex-Beispiel /instance system\.adapter\.[^\s]* started with pid [0-9]+/, // Noch ein Regex 'Start javascript script.js.', 'Stop script script.js.', 'bring.0 Cannot get translations: RequestError', ' reconnected. Old secret ', // Sonoff 'Popup-News readed...', // info.0 Adapter '[warn] Projects disabled : set editorTheme.projects.enabled=true to enable', // see https://forum.iobroker.net/topic/12260/ '[warn] Projects disabled : editorTheme.projects.enabled=false', 'Update repository "latest" under' ]; /** * Zusatz-Einstellung für Option "merge" für LOG_FILTER (unter "Konfiguration: Datenpunkte und Filter"): * In MERGE_LOGLINES_TXT kann hier ein anderes Wort eingetragen werden, z.B. 'entries' oder 'Zeilen', damit [123 entries] * oder [123 Zeilen] vorangestellt wird anstatt [123 Einträge]. * HINWEIS: Falls MERGE_LOGLINES_TXT geändert wird: bitte alle Datenpunkte des Scripts löschen und dann Script neu starten. */ const MERGE_LOGLINES_TXT = 'Einträge'; /** * Für JSON-Tabelle: Füge CSS-Klasse hinzu je nach Log-Level (debug, silly, info, warn und error), um Tabellen-Text zu formatieren. * Beispiel für Log-Level "debug": ersetzt "xxx" durch "<span class='log-debug'>xxx</span>"" * Es wird jeweils "log-" vorangestellt, also: debug -> log-debug, silly -> log-silly, info -> log-info, etc. * Etwa für Widget "basic - Table" im vis können im Reiter "CSS" z.B. folgende Zeilen hinzugefügt werden, * um Warnungen in oranger und Fehler in roter Farbe anzuzeigen. * .log-warn { color: orange; } * .log-error { color: red; } * Tipp: In LOG_FILTER kann dann bei den einzelnen Filtern mittels "jsonCssToLevel" eingestellt werden, dass das CSS * nur für die Spalte "level" (also debug, error, info) und nicht auf alle Spalten angewendet wird. */ const JSON_APPLY_CSS = true; /******************************************************************************* * Konfiguration: Datenpunkte und Filter ****************************************************************************** * Dies ist das Herzstück dieses Scripts: hier werden die Datenpunkte konfiguriert, die erstellt werden sollen. * Hierbei kannst du entsprechend Filter setzen, also z.B. Wörter/Begriffe, die in Logeinträgen enthalten sein * müssen, damit sie in den jeweiligen Datenpunkten aufgenommen werden. * -------------------------------------------------------------------------------------------------------------------------- * id: Ein Begriff ohne Leerzeichen, z.B. "error", "sonoff", homematic, etc. Die ID wird dann Teil der * Datenpunkte, z.B. "javascript.0.Log-Script.logHomematic.log" mit automatisch vorangestelltem "log". * -------------------------------------------------------------------------------------------------------------------------- * filter_all: ALLE Begriffe müssen in der Logzeile enthalten sein. Ist einer der Begriffe nicht enthalten, dann wird der * komplette Logeintrag auch nicht übernommen. Leeres Array [] eingeben, falls hier filtern nicht gewünscht. * Es ist auch Regex erlaubt. * -------------------------------------------------------------------------------------------------------------------------- * filter_any: Mindestens einer der gelisteten Begriffe muss enthalten sein. Leeres Array [] eingeben, falls hier filtern * nicht gewünscht. * Es ist auch Regex erlaubt. * -------------------------------------------------------------------------------------------------------------------------- * blacklist: Schwarze Liste: Wenn einer dieser Begriffe im Logeintrag enthalten ist, so wird der komplette Logeintrag * nicht übernommen, egal was vorher in filter_all oder filter_any definiert ist. * HINWEIS: BLACKLIST_GLOBAL wird vorher schon angewendet, hier kannst du einfach nur noch eine individuelle * Blackliste pro id definieren. * Es ist auch Regex erlaubt. * -------------------------------------------------------------------------------------------------------------------------- * clean: Der Log-Eintrag wird um diese Zeichenfolgen bereinigt, d.h. diese werden entfernt, aber die restliche Zeile * bleibt bestehen. Z.B. um unerwünschte Zeichenfolgen zu entfernen oder Log-Ausgaben zu kürzen. * Es ist auch Regex erlaubt. * -------------------------------------------------------------------------------------------------------------------------- * merge: Log-Einträge mit gleichem Text zusammenfassen. Beispiel: * ----------------------------------------------------------------------------------- * 2019-08-17 20:00:00.335 - info: javascript.0 script.js.Wetter: Wetterdaten abrufen. * 2019-08-17 20:15:00.335 - info: javascript.0 script.js.Wetter: Wetterdaten abrufen. * 2019-08-17 20:30:00.335 - info: javascript.0 script.js.Wetter: Wetterdaten abrufen. * ----------------------------------------------------------------------------------- * Daraus wird dann nur noch eine Logzeile mit letztem Datum/Uhrzeit und hinzufügen von "[3 Einträge]": * ----------------------------------------------------------------------------------- * 2019-08-17 20:30:00.335 - info: javascript.0 [3 Einträge] script.js.Wetter: Wetterdaten abrufen. * ----------------------------------------------------------------------------------- * Zum aktivieren: true eintragen, zum deaktivieren: false eintragen. * -------------------------------------------------------------------------------------------------------------------------- * sortDescending: Wenn true: Sortiert die Logeinträge absteigend, also neuester oben. * Wenn false: Sortiert die Logeinträge aufsteigend, also ältester oben. * -------------------------------------------------------------------------------------------------------------------------- * jsonColumns: Nur für JSON (für vis). * Folgende Spalten gibt es: 'date','level','source','msg'. Hier können einzelne Spalten entfernt oder die * Reihenfolge verändert werden. Bitte keine anderen Spalten eintragen, sondern nur 'date','level','source','msg'. * -------------------------------------------------------------------------------------------------------------------------- * jsonDateFormat: Datumsformat für JSON Log. Z.B. volles Datum mit 'YYYY-MM-DD hh:mm:ss' oder nur Uhrzeit mit "hh:mm:ss". Die * Platzhalter YYYY, MM, DD usw. werden jeweils ersetzt. * YYYY = Jahr 4stellig (z.B. 2019), MM = Jahr 2stellig (z.B. 20), MM = Monat, DD = Tag, hh = Stunde, mm = Minute, * ss = Sekunde. Auf Groß- und Kleinschreibung achten! * Die Verbinder (-, :, Leerzeichen, etc.) können im Prinzip frei gewählt werden. * Beispiele: 'hh:mm:ss' für 19:37:25, 'hh:mm' für 19:37, 'DD.MM. hh:mm' für '25.07. 19:37' * --- Neu seit Script-Version 4.9: * Wenn das Datum in "Hash-Zeichen (Raute = #)" gesetzt wird, dann wird es durch "Heute" bzw. "Gestern" * ersetzt. Beispiele: * - Aus '#DD.MM.# hh:mm' wird 'Heute 20:35', falls der Log von heute ist. * - Aus '#DD.MM.YYYY# hh:mm' wird 'Gestern 20:35', falls der Log von gestern ist. * - Aus '#DD.MM.YYYY# hh:mm' wird '18.02. 20:35', falls der Log nicht von heute oder gestern ist. * - Aus '#DD.MM.YY# hh:mm' wird '18.02.20 20:35', falls der Log nicht von heute oder gestern ist. * -------------------------------------------------------------------------------------------------------------------------- * jsonLogLength: Maximale Anzahl Zeichen jeder einzelnen Log-Meldung im JSON-Log. Alles was länger ist, wird abgeschnitten. * -------------------------------------------------------------------------------------------------------------------------- * jsonMaxLines: Maximale Anzahl der letzten Logeinträge im JSON-Log. Alle älteren werden entfernt. * Falls in MAX_LOG_LINES z.B. "100" gesetzt wird, wird hier bei 100 der Cut gemacht, selbst wenn in * jsonMaxLines etwa 250 eingetragen wird. D.h. im Bedarf zuerst MAX_LOG_LINES anpassen/erhöhen. * -------------------------------------------------------------------------------------------------------------------------- * jsonCssToLevel: Wenn true, dann wird JSON_APPLY_CSS nur für die Spalte "level" (also debug, error, info) angewendet, * aber nicht für die restlichen Spalten wie Datum, Log-Eintrag, etc. Falls alle Spalten das CSS bekommen sollen: auf false setzen. * -------------------------------------------------------------------------------------------------------------------------- * * WEITERER HINWEIS: * Bestehende Datenpunkt-Inhalte dieses Scripts bei Anpassung dieser Option werden nicht nachträglich neu * gefiltert, sondern nur alle neu hinzugefügten Log-Einträge ab Speichern des Scripts werden berücksichtigt. * -------------------------------------------------------------------------------------------------------------------------- */ const LOG_FILTER = [ // Beispiel für individuellen Eintrag. Hier wird euer Hubschrauber-Landeplatz überwacht :-) Wir wollen nur Einträge // vom Adapter 'hubschr.0'. Dabei sollen entweder Wetterwarnungen, Alarme, oder UFOs gemeldet werden. Alles unter // Windstärke "5 Bft" interessiert uns dabei nicht, daher haben wir '0 Bft' bis '4 Bft' auf die Blackliste gesetzt. // Außerdem entfernen wir von der Log-Zeile die Zeichenfolgen '****', '!!!!' und 'ufo gesichtet', der Rest bleibt // aber bestehen. Zudem haben wir unter jsonColumns die Spaltenreihenfolge geändert. 'level' herausgenommen, und Quelle // ganz vorne. /* { id: 'hubschrauberlandeplatz', filter_all: ['hubschr.0'], filter_any: ['wetterwarnung', 'alarm', 'ufo'], blacklist: ['0 Bft', '1 Bft', '2 Bft', '3 Bft', '4 Bft'], clean: ['****', '!!!!', 'ufo gesichtet'], merge: true, sortDescending: true, jsonDateFormat: '#DD.MM.# hh:mm', jsonColumns: ['source','date','msg'], jsonLogLength: 100, jsonMaxLines: 10, jsonCssToLevel: true, }, */ /* { id: 'all', // Beispiel "all": hier kommen alle Logeinträge rein, keine Filterung filter_all: ['', ''], // wird ignoriert, wenn leer filter_any: ['', ''], // wird ignoriert, wenn leer blacklist: ['', ''], // wird ignoriert, wenn leer clean: ['', '', ''], // wird ignoriert, wenn leer merge: true, sortDescending: true, jsonDateFormat: '#DD.MM.# hh:mm', jsonColumns: ['date','level','source','msg'], // Spaltenreihenfolge für JSON (Tabelle in vis) jsonLogLength: 100, jsonMaxLines: 10, jsonCssToLevel: true, }, */ { id: 'info', filter_all: [' - info: '], // nur Logeinträge mit Level 'info' filter_any: ['', ''], blacklist: ['', '', '', ' [Alexa-Log-Script]'], ///##\{\s?\".*\"\s?\}##/, clean: [/script\.js\.[^:]*: /, ''], merge: true, sortDescending: true, jsonDateFormat: '#DD.MM.# hh:mm', jsonColumns: ['date','level','source','msg'], jsonLogLength: 100, jsonMaxLines: 100, jsonCssToLevel: true, }, { id: 'warn', filter_all: [' - warn: ', ''], // nur Logeinträge mit Level 'warn' filter_any: [''], blacklist: ['', '', ''], clean: [/script\.js\.[^:]*: /, '', ''], merge: false, sortDescending: true, jsonColumns: ['date','level','source','msg'], jsonDateFormat: '#DD.MM.# hh:mm', jsonLogLength: 100, jsonMaxLines: 100, jsonCssToLevel: true, }, { id: 'error', filter_all: [' - error: ', ''], // nur Logeinträge mit Level 'error' filter_any: [''], blacklist: ['', '', ''], clean: [/script\.js\.[^:]*: /, '', ''], merge: true, sortDescending: true, jsonColumns: ['date','level','source','msg'], jsonDateFormat: '#DD.MM.# hh:mm', jsonLogLength: 100, jsonMaxLines: 100, jsonCssToLevel: true, }, /* { id: 'warnanderror', filter_all: ['', ''], filter_any: [' - error: ', ' - warn: '], // nur Logeinträge mit Levels 'warn' und 'error' blacklist: ['no playback content', 'Ignore! Actual secret is '], clean: [/script\.js\.[^:]*: /, '', ''], merge: true, sortDescending: true, jsonDateFormat: '#DD.MM.# hh:mm', jsonColumns: ['date','level','source','msg'], jsonLogLength: 100, jsonMaxLines: 100, jsonCssToLevel: true, }, */ { id: 'alexa', filter_all: ['[Alexa-Log-Script]', ''], filter_any: [' - info: '], blacklist: ['', '', ''], clean: [/script\.js\.[^:]*: [Alexa-Log-Script]/, '', ''], merge: false, sortDescending: true, jsonDateFormat: '#DD.MM.# hh:mm', jsonColumns: ['date','level','source','msg'], jsonLogLength: 100, jsonMaxLines: 50, jsonCssToLevel: true, }, // Beispiel, um Adapter zu überwachen. // Hier werden alle Fehler und Warnungen der Homematic-Adapters hm-rpc.0 und hm-rega.0 gelistet. /* id: 'homematic', filter_all: [/hm-(rpc|rega)\.[0-9]/], filter_any: [' - error: ', ' - warn: ', ' - info: '], // entweder error oder warn blacklist: ['', '', ''], clean: ['', '', ''], merge: true, sortDescending: true, jsonDateFormat: '#DD.MM.# hh:mm', jsonColumns: ['date','level','source','msg'], jsonLogLength: 100, jsonMaxLines: 100, jsonCssToLevel: true, }, */ ]; /******************************************************************************* * Optionale Konfiguration: Auswahl Log-Filter für VIS unterhalb ".All" ******************************************************************************/ // Wenn man in VIS eine Tabelle der Logs ausgibt, kann man hiermit mit Buttons // ('Homematic', 'Warnungen', 'Fehler' usw. zwischen den einzelnen Filtern umschalten, // die dann dynamisch jeweils in dieser einen Tabelle ausgegeben werden. // Hier die Anzahl der unterschiedlichen VIS-Views angeben, in denen du das brauchst. // Diese werden angelegt unter '.All.visView1', '.All.visView2', usw. // Falls auf 0 gestellt, dann werden gar keine Datenpunkte ausgegeben. const NUMBER_OF_VIS_VIEWS = 1; /******************************************************************************* * Konfiguration: Konsolen-Ausgaben ******************************************************************************/ // Auf true setzen, wenn zur Fehlersuche einige Meldungen ausgegeben werden sollen. // Ansonsten bitte auf false stellen. const SCRIPT_DEBUG = false; // Auf true setzen, wenn ein paar Infos dieses Scripts im Log ausgegeben werden dürfen, bei false bleiben die Infos komplett weg. const LOG_INFO = true; /******************************************************************************* * Experten-Konfiguration ******************************************************************************/ // Wie oft Datenpunkte aktualisieren? // Neu reinkommende Logeinträge werden erst mal gesammelt (in Variable G_NewLogLinesArrayToProcess). Diese werden dann // regelmäßig in den Datenpunkten geschrieben. Sinnvoll ist hier nicht kürzer als 2-3 Sekunden, und nicht länger als // ein paar Minuten. Zu kurzes Intervall: Script kommt nicht mehr nach. Zu lange: falls viele Logeinträge reinkommen, // kann sich vieles "aufstauen" zur Abarbeitung. Benutze den "Cron"-Button oben rechts für komfortable Einstellung. const STATE_UPDATE_SCHEDULE = '*/20 * * * * *'; // alle 20 Sekunden // JSON-Tabelle: ersetze heutiges und gestriges Datum durch 'Heute' bzw. 'Gestern'. // Mittels Hash-Zeichen(#) kann in LOG_FILTER in der Option "jsonDateFormat" definiert werden, // dass heutiges und gestriges Datum durch 'Heute' bzw. 'Gestern' ersetzt wird. // Hier können andere Begriffe statt "Heute"/"Gestern" definiert werden. const TXT_TODAY = 'Heute'; const TXT_YESTERDAY = 'Gestern'; // Leer lassen! Nur setzen, falls ein eigener Filename für das Logfile verwendet wird für Debug. const DEBUG_CUSTOM_FILENAME = ''; // Debug: Ignore. Wenn dieses String in der Logzeile enthalten ist, dann ignorieren wir es. // Dient dazu, dass wir während des Scripts ins Log schreiben können, ohne dass das dieses Script berücksichtigt. const DEBUG_IGNORE_STR = '[LOGSCRIPT_IGNORE]'; // Muss ein individuelles String sein. Sonst gibt es ggf. eine Endlos-Schleife. // Debug: Prüfen, ob jede Logzeile erfasst wird, in dem wir diese direkt danach noch mal ins Log schreiben. // Bitte nur auf Anweisung vom Entwickler einschalten. Sonst wird jeder Logeintrag noch einmal wiederholt, // mit führendem DEBUG_EXTENDED_STR am Anfang und max. DEBUG_EXTENDED_NO_OF_CHARS an Anzahl Zeichen. const DEBUG_EXTENDED = false; const DEBUG_EXTENDED_STR = '[LOGSCRIPT_DEBUG_EXTENDED]'; // Muss ein individuelles String sein. Sonst gibt es ggf. eine Endlos-Schleife. const DEBUG_EXTENDED_NO_OF_CHARS = 120; /************************************************************************************************************************* * Ab hier nichts mehr ändern / Stop editing here! *************************************************************************************************************************/ /************************************************************************************************************************* * Global variables and constants *************************************************************************************************************************/ // Final state path const FINAL_STATE_LOCATION = validateStatePath(LOG_STATE_PATH, false); const FINAL_STATE_PATH = validateStatePath(LOG_STATE_PATH, true); // Regex für die Aufteilung des Logs in 1-Datum/Zeit, 3-Level, 5-Quelle und 7-Logtext. // Ggf. anzupassen bei anderem Datumsformat im Log. Wir erwarten ein Format // wie z.B.: '2018-07-22 12:45:02.769 - info: javascript.0 Stop script script.js.ScriptAbc' // Da als String, wurden alle Backslashes "\" mit einem zweiten Backslash escaped. const LOG_PATT = '([0-9_.\\-:\\s]*)(\\s+\\- )(silly|debug|info|warn|error|)(: )([^\\s]*)(\\s)(.*)'; // Final number of VIS states. We do not allow more than 9. function finalVisStatesQty(qty) { let result = 0; if( (!isLikeEmpty(qty)) && (qty > 0) ) { if(qty > 9) { result = 9; } else { result = qty; } } return result; }; const FINAL_NUM_OF_VIS_VIEWS = finalVisStatesQty(NUMBER_OF_VIS_VIEWS); // Merge loglines: define pattern (and escape the merge text) // We added an additional backslash '\' to each backslash as these need to be escaped. const MERGE_REGEX_PATT = '^\[(\\d+)\\s' + escapeRegExp(MERGE_LOGLINES_TXT) + '\]\\s(.*)'; // Log Handler variable for ioBroker function onLog() let G_LogHandler; // being set later // Schedule for logfile update let G_Schedule_StateUpdate; // being set later // Schedule for midnight JSON update let G_Schedule_JsonUpdate; // being set later // We add here all the new log lines to be processed regularly (per STATE_UPDATE_SCHEDULE); let G_NewLogLinesArrayToProcess = []; /************************************************************************************************************************* * init - This is executed on every script (re)start. *************************************************************************************************************************/ init(); function init() { // Unsubscribe log handler onLogUnregister(G_LogHandler); // Create script states createLogScriptStates(function() { // force = false // -- All states created, so we continue by using callback // Subscribe on changes: Pressed button "clearJSON" subscribeClearJson(); // Subscribe to log handler G_LogHandler = onLog('*', data => { // please ignore the red squiggly underline under '*', see Github issue: https://github.com/ioBroker/ioBroker.javascript/issues/457 processNewLogLine(data); }); // Schedule writing changes into states clearSchedule(G_Schedule_StateUpdate); G_Schedule_StateUpdate = schedule(STATE_UPDATE_SCHEDULE, processNewLogsPerSchedule); // Schedule midnight Json update (due to "Heute"/"Gestern" in date) clearSchedule(G_Schedule_JsonUpdate); G_Schedule_JsonUpdate = schedule('1 0 * * *', function() { // Um 00:01 jeden Tag processLogArrayAndSetStates([], true); // update JSON only }); // Subscribe to clear all states stateSubscribeClearAllJSON() // Subscribe to states for the VIS views if(FINAL_NUM_OF_VIS_VIEWS > 0) { stateSubscribeVisWhichFilter(); // which Filter stateSubscribeVisClearJson(); // clear JSON } // Message if (LOG_INFO) log('Start monitoring of the ioBroker log...', 'info'); // Update JSON setTimeout(function(){ processLogArrayAndSetStates([], true); // update JSON only }, 5000); }); } function processNewLogLine(data) { // Convert to Log Line // TODO: This is a quick implementation of new function onLog(). // We need to entirely rewrite script later to fully use the data object. // However, at this time, we convert it to a standard log line being expected. // First, remove PID if desired let msg = data.message; if (REMOVE_PID) msg = removePID(msg); // Now convert to log line let newLogEntry = timestampToLogDate(data.ts) + ' - [32m' + data.severity + '[39m: ' + msg; // Check if we have DEBUG_IGNORE_STR in the new log line if(! newLogEntry.includes(DEBUG_IGNORE_STR)) { if (newLogEntry.length > 45) { // a log line with less than 45 chars is not a valid log line. // Cleanse and apply blacklist newLogEntry = cleanseLogLine(newLogEntry); // some debugging // Important: You will need to change onLog('*') to a certain severity so that this will work. // See onLog() Documentation: "Important: you cannot output logs in handler with the same severity to avoid infinite loops." if (SCRIPT_DEBUG) log(DEBUG_IGNORE_STR + '===============================================================', 'warn'); if (SCRIPT_DEBUG) log(DEBUG_IGNORE_STR + 'New Log Entry, Len (' + newLogEntry.length + '), content: [' + newLogEntry + ']', 'warn'); // Push result into logArrayFinal G_NewLogLinesArrayToProcess.push(newLogEntry); // This is for debugging purposes, and it will log every new log entry once again. See DEBUG_EXTENDED option above. if (DEBUG_EXTENDED) { if (! newLogEntry.includes(DEBUG_EXTENDED_STR)) { // makes sure no endless loop here. log(DEBUG_EXTENDED_STR + newLogEntry.substring(0, DEBUG_EXTENDED_NO_OF_CHARS)); } } } } } /** * Called per schedule STATE_UPDATE_SCHEDULE. * It processes G_NewLogLinesArrayToProcess */ function processNewLogsPerSchedule() { if (! isLikeEmpty (G_NewLogLinesArrayToProcess) ) { // We use array spreads '...' to copy array. If not, array is changed by reference and not value. // That means, if we change the target array, it will also change the source array. // See https://stackoverflow.com/questions/7486085/copy-array-by-value let logArrayToProcess = [...G_NewLogLinesArrayToProcess]; G_NewLogLinesArrayToProcess.length = 0; // emptying array. https://stackoverflow.com/questions/4804235/difference-between-array-length-0-and-array /** * Apply the filters as set in LOG_FILTER and split up log levels into elements of an array * logArrayToProcessFiltered will look as follows: * logArrayToProcessFiltered = [ * ['info':'15.08.2019 09:27:55.476 info adapt.0 some log', 'error':''], * ['info':'15.08.2019 09:33:58.522 info adapt.0 some more log', 'error':''], * ['info':'', 'error':'15.08.2019 09:37:55.807 error adapt.0 some error log'] * ] */ let logArrayToProcessFiltered = []; for (let lpEntry of logArrayToProcess) { let logEntryFilteredArray = applyFilter(lpEntry); logArrayToProcessFiltered.push(logEntryFilteredArray); } // Further process and finally set states with our results. processLogArrayAndSetStates(logArrayToProcessFiltered); } // Finally, set updatestate with current date/time setState(FINAL_STATE_PATH + '.All.lastTimeUpdated', Date.now()); } /************************************************************************************************************************* * Filtering *************************************************************************************************************************/ /** * This function applies the filters as set in LOG_FILTER. * Also, it splits up the log levels into elements of an array we return by this function. * @param {string} strLogEntry * @return {array} split up log levels as elements within this array, like: ['info':'logtext', 'error':'logtext'] etc. */ function applyFilter(strLogEntry) { // We add one element per each filter to the Array ('all', 'error', etc.) let logArrayProcessed = []; for (let j = 0; j < LOG_FILTER.length; j++) { logArrayProcessed[LOG_FILTER[j].id] = ''; } // We apply regex here. This will also eliminate all log lines without proper info // like date/time, log level, and entry. let arrSplitLogLine = logLineSplit(strLogEntry); if (arrSplitLogLine !== false) { if (isLikeEmpty(LOG_FILTER) === false) { // Now let's iterate over the filter array elements // We check if both the "all" and "any" filters apply. If yes, - and blacklist false - we add the log line. for (let k = 0; k < LOG_FILTER.length; k++) { if ( (stringMatchesList(strLogEntry, LOG_FILTER[k].filter_all, true) === true) && (stringMatchesList(strLogEntry, LOG_FILTER[k].filter_any, false) === true) && (isBlacklisted(strLogEntry, LOG_FILTER[k].blacklist) === false) ) { logArrayProcessed[LOG_FILTER[k].id] = logArrayProcessed[LOG_FILTER[k].id] + strLogEntry + "\n"; } // Now we remove terms if desired per "clean" option if (isLikeEmpty(LOG_FILTER[k].clean) === false) { for (let lpTerm of LOG_FILTER[k].clean) { if (! isLikeEmpty(lpTerm)) { logArrayProcessed[LOG_FILTER[k].id] = logArrayProcessed[LOG_FILTER[k].id].replace(lpTerm, ''); } } } } } } return logArrayProcessed; } /************************************************************************************************************************* * Further processing *************************************************************************************************************************/ /** * Further processes the log array and set states accordingly. * * @param {array} arrayLogInput The Array of the log input. * Array is like: * [ * ['info':'15.08.2019 09:27:55.476 info adapt.0 some log', 'error':''], * ['info':'15.08.2019 09:33:58.522 info adapt.0 some more log', 'error':''], * ['info':'', 'error':'15.08.2019 09:37:55.807 error adapt.0 some error log'], * ] * @param {boolean} [updateJsonOnly=false] Optional parameter. If true, then just the JSON states will be updated without * considering arrayLogInput. Used for updated JSON every midnight due to texts * "Today"/"Yesterday" in JSON table. **/ function processLogArrayAndSetStates(arrayLogInput, updateJsonOnly) { let justUpdateJson = (typeof updateJsonOnly == 'undefined') ? false : updateJsonOnly; /***************** * [1] Build array from LOG_FILTER. Looks like: arrayFilterIds = ['info', 'error', 'warn']. * Also, build result array to keep our results. Lools like resultArr = [info: '', error: '', warn: ''] *****************/ let arrayFilterIds = []; let resultArr = []; for (let i = 0; i < LOG_FILTER.length; i++) { arrayFilterIds.push(LOG_FILTER[i].id); // each LOG_FILTER id into array resultArr[LOG_FILTER[i].id] = ''; } if (! justUpdateJson) { /***************** * [2] Process element by element, so ['info':'log test', 'error':'log test'] of given array. * We fill the result array accordingly. *****************/ for (let lpElement of arrayLogInput) { // Loop thru our new array arrayFilterIds and fill result array for (let k = 0; k < arrayFilterIds.length; k++) { // some variables let lpFilterId = arrayFilterIds[k]; // Filter ID from LOG_FILTER, like 'error', 'info', 'custom', etc. let lpNewLogLine = lpElement[lpFilterId]; // Current log line of provided array element of 'error', 'info', 'custom' etc. if (isLikeEmpty(lpNewLogLine)) { // No log content for the given filter id. if (SCRIPT_DEBUG) log (DEBUG_IGNORE_STR + 'Filter [' + lpFilterId + ']: No match.'); } else { if (SCRIPT_DEBUG) log (DEBUG_IGNORE_STR + 'Filter [' + lpFilterId + ']: Match! New Log Line length: (' + lpNewLogLine.length + ')'); // Append new log line to result array if (isLikeEmpty(resultArr[lpFilterId])) { resultArr[lpFilterId] = lpNewLogLine; } else { resultArr[lpFilterId] = lpNewLogLine + resultArr[lpFilterId]; // "\n" not needed, always added above } } } } } /***************** * [3] We merge with the current state. *****************/ for (let k = 0; k < arrayFilterIds.length; k++) { let lpFilterId = arrayFilterIds[k]; // Filter ID from LOG_FILTER, like 'error', 'info', 'custom', etc. let lpStatePath1stPart = FINAL_STATE_PATH + '.log' + cleanseStatePath(lpFilterId); // Get Path to state let lpNewFinalLog = resultArr[lpFilterId]; if ( justUpdateJson || (! isLikeEmpty(lpNewFinalLog)) ) { // Get state value let strCurrentStateLog = getState(lpStatePath1stPart + '.log').val; // Get state contents of loop item // Add state log lines to our final log if ( !isLikeEmpty(strCurrentStateLog)) { if (! justUpdateJson) { lpNewFinalLog = lpNewFinalLog + strCurrentStateLog; // "\n" not needed, always added above } else { lpNewFinalLog = strCurrentStateLog; } } // Convert to array for easier handling let lpNewFinalLogArray = lpNewFinalLog.split(/\r?\n/); // Remove duplicates lpNewFinalLogArray = arrayRemoveDublicates(lpNewFinalLogArray); // Remove empty values lpNewFinalLogArray = cleanArray(lpNewFinalLogArray); // Sort array descending lpNewFinalLogArray = sortLogArrayByDate(lpNewFinalLogArray, 'desc'); // Merge Loglines if multiple values and add leading '[123 entries]' to log message let doMerge = getConfigValuePerKey(LOG_FILTER, 'id', lpFilterId, 'merge'); if (doMerge || doMerge === 'true') { // also check for string 'true' in case user used string lpNewFinalLogArray = mergeLogLines(lpNewFinalLogArray); } // We need a separate array for JSON let lpNewFinalLogArrayJSON = lpNewFinalLogArray; // Let's remove elements if time of when button '.clearJSON' was pressed is greater than log date. lpNewFinalLogArrayJSON = clearJsonByDate(lpNewFinalLogArrayJSON, lpStatePath1stPart + '.clearJSON'); // Just keep the first x elements of the log. JSON log length is being set individually. lpNewFinalLogArray = lpNewFinalLogArray.slice(0, MAX_LOG_LINES); lpNewFinalLogArrayJSON = lpNewFinalLogArrayJSON.slice(0, getConfigValuePerKey(LOG_FILTER, 'id', lpFilterId, 'jsonMaxLines')); // Get just the most recent log entry into string let lpMostRecent = lpNewFinalLogArray[0]; // Sort ascending if desired if (!getConfigValuePerKey(LOG_FILTER, 'id', lpFilterId, 'sortDescending')) { lpNewFinalLogArray = lpNewFinalLogArray.reverse(); lpNewFinalLogArrayJSON = lpNewFinalLogArrayJSON.reverse(); } // ** Finally set the states /////////////////////////////// // -1- Full Log, String, separated by "\n" /////////////////////////////// if (! justUpdateJson) { let strFullLog = lpNewFinalLogArray.join("\n"); if (SCRIPT_DEBUG) log (DEBUG_IGNORE_STR + 'New length to be set into state: (' + strFullLog.length + '), state: [' + lpStatePath1stPart + '.log' + ']'); setState(lpStatePath1stPart + '.log', strFullLog); } /////////////////////////////// // -2- JSON, with elements date and msg /////////////////////////////// // Let's put together the JSON let jsonArr = []; for (let j = 0; j < lpNewFinalLogArrayJSON.length; j++) { // Get 4 elements in array: datetime, level, source, message let arrSplitLogLine = logLineSplit(lpNewFinalLogArrayJSON[j]); if (arrSplitLogLine !== false) { let strLogMsg = arrSplitLogLine.message; // Reduce the length for each log message per "jsonLogLength" strLogMsg = strLogMsg.substr(0, LOG_FILTER[k].jsonLogLength); // ++++++ // Build the final Array // ++++++ // We need this section to generate the JSON with the columns (which ones, and order) as specified in LOG_FILTER let objectJSONentry = {}; // object (https://stackoverflow.com/a/13488998) if (isLikeEmpty(LOG_FILTER[k].jsonColumns)) log('Columns not specified in "jsonColumns".', 'warn'); // Prepare CSS let strCSS1, strCSS2; let strCSS1_level, strCSS2_level; if (JSON_APPLY_CSS) { strCSS1 = "<span class='log-" + arrSplitLogLine.level + "'>"; strCSS2 = '</span>'; strCSS1_level = strCSS1; strCSS2_level = strCSS2; if (LOG_FILTER[k].jsonCssToLevel) { strCSS1 = ''; strCSS2 = ''; } } /** * Support individual items in column provided through log * Syntax: "Das ist ein Log ##{"msg":"Individuell", "source":"andere Quelle"}##" */ // Check if we actually have a hit let individualRegexArr = strLogMsg.match(/##(\{\s?\".*\"\s?\})##/); let individualRegexObjRes; if (individualRegexArr != null) { if(individualRegexArr[1] != undefined) { individualRegexObjRes = JSON.parse(individualRegexArr[1]); } else { individualRegexObjRes = null; } } else { individualRegexObjRes = null; } for (let lpCol of LOG_FILTER[k].jsonColumns) { switch (lpCol) { case 'date' : let caseDate = formatLogDateStr(arrSplitLogLine.datetime, LOG_FILTER[k].jsonDateFormat); if( (individualRegexObjRes != null) && (individualRegexObjRes['date'] != undefined) ) { caseDate = individualRegexObjRes['date']; } objectJSONentry.date = strCSS1 + caseDate + strCSS2; break; case 'level' : let caseLevel = arrSplitLogLine.level; if( (individualRegexObjRes != null) && (individualRegexObjRes['level'] != undefined) ) { caseLevel = individualRegexObjRes['level']; } objectJSONentry.level = strCSS1_level + caseLevel + strCSS2_level; break; case 'source' : let caseSource = arrSplitLogLine.source; if( (individualRegexObjRes != null) && (individualRegexObjRes['source'] != undefined) ) { caseSource = individualRegexObjRes['source']; } objectJSONentry.source = strCSS1 + caseSource + strCSS2; break; case 'msg' : let caseMsg = strLogMsg; if( (individualRegexObjRes != null) && (individualRegexObjRes['msg'] != undefined) ) { caseMsg = individualRegexObjRes['msg']; } objectJSONentry.msg = strCSS1 + caseMsg + strCSS2; break; default: //nothing; } } // Ok, so now we have the JSON entry. jsonArr.push(objectJSONentry); } } if (! isLikeEmpty(lpNewFinalLogArrayJSON)) { setState(lpStatePath1stPart + '.logJSON', JSON.stringify(jsonArr)); setState(lpStatePath1stPart + '.logJSONcount', lpNewFinalLogArrayJSON.length); } } } /***************** * [4] Finally done. Now we set states for the VIS views. *****************/ if(FINAL_NUM_OF_VIS_VIEWS > 0) { setTimeout(function() { // Apply timeout, to ensure all states are set. for(let i = 0; i < FINAL_NUM_OF_VIS_VIEWS; i++) { let lpStateViewSelFilter = FINAL_STATE_PATH + '.All.visView' + (i+1); // like: 0_userdata.0.Log-Script.All.visView1.whichFilter let currentVisFilter = getState(lpStateViewSelFilter + '.whichFilter').val; if(! isLikeEmpty(currentVisFilter)) { let jsonStateToGet = LOG_STATE_PATH + '.log' + currentVisFilter + '.logJSON'; if( isState(jsonStateToGet) ) { setState(lpStateViewSelFilter + '.outputJSON', getState(jsonStateToGet).val); // Finally: set state setState(lpStateViewSelFilter + '.outputJSONcount', getState(jsonStateToGet + 'count').val); // Finally: set state } else { log('Log-Script-Fehler: Gewählter Filter ' + currentVisFilter + ', aber Datenpunkt [' + jsonStateToGet + '] ist nicht vorhanden.', 'error'); } } } }, 2000); } } /************************************************************************************************************************* * Script specific supporting functions *************************************************************************************************************************/ /** * Subscribe to all Log-Script.logXXXX.clearJSON * This will allow to clear JSON log if button is pressed. */ function subscribeClearJson() { // Set current date to state if button is pressed let logSubscribe = ''; for (let i = 0; i < LOG_FILTER.length; i++) { let lpFilterId = cleanseStatePath(LOG_FILTER[i].id); let lpStateFirstPart = FINAL_STATE_PATH + '.log' + lpFilterId; logSubscribe += ( (logSubscribe === '') ? '' : ', ') + lpFilterId; on({id: lpStateFirstPart + '.clearJSON', change: 'any', val: true}, function(obj) { let stateBtnPth = obj.id // e.g. [javascript.0.Log-Script.logInfo.clearJSON] let firstPart = stateBtnPth.substring(0, stateBtnPth.length-10); // get first part of obj.id, like "javascript.0.Log-Script.logInfo" let filterID = firstPart.slice(firstPart.lastIndexOf('.') + 1); // gets the filter id, like "logInfo" if (SCRIPT_DEBUG) log(DEBUG_IGNORE_STR + 'Clear JSON states for [' + filterID + '].'); // We clear the according JSON states setState(firstPart + '.logJSON', '[]'); setState(firstPart + '.logJSONcount', 0); }); } if (SCRIPT_DEBUG) log (DEBUG_IGNORE_STR + 'Subscribing to Clear JSON Buttons: ' + logSubscribe) } /** * Subscribe to all .All.clearAllJSON */ function stateSubscribeClearAllJSON() { on({id: FINAL_STATE_PATH + '.All.clearAllJSON', change: 'any', val: true}, function (obj) { // 1. All fitered log states const ALL_FILTER_IDS = getAllFilterIds(); for (let loopFilterId of ALL_FILTER_IDS) { // Version 4.5.6 // See https://forum.iobroker.net/post/375932 // We do no longer set the target states directly, but use the according clearJSON instead. // This is to ensure that user can use the clearJSON timestamp for the time of last clearJSON. setState(FINAL_STATE_PATH + '.log' + loopFilterId + '.clearJSON', true); // setState(FINAL_STATE_PATH + '.log' + loopFilterId + '.logJSON', '[]'); // setState(FINAL_STATE_PATH + '.log' + loopFilterId + '.logJSONcount', 0); } // 2. Vis add-ons if(FINAL_NUM_OF_VIS_VIEWS > 0) { for(let i = 0; i < FINAL_NUM_OF_VIS_VIEWS; i++) { let lpState = FINAL_STATE_PATH + '.All.visView' + (i+1); setState(lpState + '.outputJSON', '[]'); setState(lpState + '.outputJSONcount', 0); } } }); } /** * Subscribe to .All.visViewX.whichFilter */ function stateSubscribeVisWhichFilter() { if(FINAL_NUM_OF_VIS_VIEWS > 0) { for(let i = 0; i < FINAL_NUM_OF_VIS_VIEWS; i++) { let lpState = FINAL_STATE_PATH + '.All.visView' + (i+1); on({id: lpState + '.whichFilter', change: 'ne'}, function (obj) { if(!isLikeEmpty(obj.state.val)) { let firstStatePart = obj.id.substr(0, (obj.id.length - obj.id.split('.').pop().length - 1)); // Gibt von '0_userdata.0.System.Log-Script.All.visView1.whichFilter' nur den Hauptteil ohne String nach letztem Punkt zurück, also "0_userdata.0.System.Log-Script.All.visView1" setState(firstStatePart + '.' + 'outputJSON', getState(LOG_STATE_PATH + '.log' + obj.state.val + '.logJSON').val); setState(firstStatePart + '.' + 'outputJSONcount', getState(LOG_STATE_PATH + '.log' + obj.state.val + '.logJSONcount').val); } }); } } } /** * Subscribe to all .All.visViewX.clearJSON */ function stateSubscribeVisClearJson() { if(FINAL_NUM_OF_VIS_VIEWS > 0) { for(let i = 0; i < FINAL_NUM_OF_VIS_VIEWS; i++) { let lpState = FINAL_STATE_PATH + '.All.visView' + (i+1); on({id: lpState + '.clearJSON', change: 'any', val: true}, function (obj) { let firstStatePart = obj.id.substr(0, (obj.id.length - obj.id.split('.').pop().length - 1)); // Gibt von '0_userdata.0.System.Log-Script.All.visView1.clearJSON' nur den Hauptteil ohne String nach letztem Punkt zurück, also "0_userdata.0.System.Log-Script.All.visView1" let selectedFilter = getState(firstStatePart + '.whichFilter').val; if(!isLikeEmpty(selectedFilter)) { setState(firstStatePart + '.outputJSON', '[]'); // .All.visViewX.outputJSON leeren setState(firstStatePart + '.outputJSONcount', 0); // .All.visViewX.outputJSONcount leeren setState(FINAL_STATE_PATH + '.log' + selectedFilter + '.clearJSON', true); // Log-Script.logXXXX.clearJSON leeren } }); } } } /** * Get all filter ids into an array (Homematic, Warnanderror, etc.) * @return [array] Filter ids in Array */ function getAllFilterIds() { let allFilterIds = []; for(let i = 0; i < LOG_FILTER.length; i++) { let lpIDClean = cleanseStatePath(LOG_FILTER[i].id) if (LOG_FILTER[i].id !== '') allFilterIds.push(lpIDClean); // für VIS add-on states }; return allFilterIds; } /** * Reformats a log date string accordingly. * @param {string} strDate The date to convert, e.g. 2018-07-22 11:47:53.019 * @param {string} format Like 'yyyy-mm-dd HH:MM:SS'. Both upper case and lower case letters are allowed. * If date is within hash (#), so like '#yyyy-mm-dd# HH:MM:SS', it will be replaced * with "Heute"/"Gestern" if date is today/yesterday. * @return {string} Returns the resulting date string */ function formatLogDateStr(strDate, format) { const TXT_TODAY_FC = ( typeof TXT_TODAY !== 'undefined' ) ? TXT_TODAY : 'Heute'; // Version 4.9: kann man später rauswerfen. Ist nur da falls User von früherer Version updaten und diese Konfig nicht haben. const TXT_YESTERDAY_FC = ( typeof TXT_YESTERDAY !== 'undefined' ) ? TXT_YESTERDAY : 'Gestern'; // Version 4.9: kann man später rauswerfen. Ist nur da falls User von früherer Version updaten und diese Konfig nicht haben. let strResult = format; // 1. Replace today's date and yesterday's date with TXT_TODAY_FC / TXT_YESTERDAY_FC let hashkMatch = strResult.match(/#(.*)#/); if (hashkMatch != null) { let todayYesterdayTxt = todayYesterday(new Date(strDate)); if(todayYesterdayTxt != '') { // We have either today or yesterday, so set according txt strResult = strResult.replace('#'+hashkMatch[1]+'#', todayYesterdayTxt); } else { // Neither today nor yesterday, so remove all ## strResult = strResult.replace(/#/g, ''); } } // 2. Replace all the rest strResult = strResult.replace('YYYY', strDate.substr(0,4)); strResult = strResult.replace('YY', strDate.substr(2,2)); strResult = strResult.replace('MM', strDate.substr(5,2)); strResult = strResult.replace('DD', strDate.substr(8,2)); strResult = strResult.replace('hh', strDate.substr(11,2)); strResult = strResult.replace('mm', strDate.substr(14,2)); strResult = strResult.replace('ss', strDate.substr(17,2)); return strResult; /** * @param {object} dateGiven Date object, created with new Date() * @return 'Heute', if today, 'Gestern' if yesterday, empty string if neither today nor yesterday */ function todayYesterday(dateGiven) { const today = new Date(); const yesterday = new Date(); yesterday.setDate(today.getDate() - 1) if (dateGiven.toLocaleDateString() == today.toLocaleDateString()) { return TXT_TODAY_FC; } else if (dateGiven.toLocaleDateString() == yesterday.toLocaleDateString()) { return TXT_YESTERDAY_FC; } else { return ''; } } } /** * Cleanse the log line * @param {string} logLine The log line to be cleansed. * @return {string} The cleaned log line */ function cleanseLogLine(logLine) { // Remove color escapes - https://stackoverflow.com/questions/25245716/remove-all-ansi-colors-styles-from-strings let logLineResult = logLine.replace(/\u001b[.*?m/g, ''); // Sometimes, a log line starts with the term "undefined", so we remove it. if (logLineResult.substr(0,9) === 'undefined') logLineResult = logLineResult.substr(9); // Remove white space, tab stops, new line logLineResult = logLineResult.replace(/\s\s+/g, ' '); // Check against global blacklist if(isBlacklisted(logLineResult, BLACKLIST_GLOBAL)) logLineResult = ''; return logLineResult; } /** * Sorts the log array by date. We expect the first 23 chars of each element being a date in string format. * @param {array} inputArray Array to process * @param {string} order 'asc' or 'desc' for ascending or descending order */ function sortLogArrayByDate(inputArray, order) { var result = inputArray.sort(function(a,b){ // Turn your strings into dates, and then subtract them // to get a value that is either negative, positive, or zero. a = new Date(a.substr(0,23)); b = new Date(b.substr(0,23)); if (order === 'asc') { return a - b; } else { return b - a; } }); return result; } /** * Splits a given log entry into an array with 4 elements. * @param {string} inputValue Log line like '2018-07-22 11:47:53.019 - info: javascript.0 script.js ...' * @return {object} Array with 4 elements: * 0. datetime (e.g. 2018-07-22 11:47:53.019), * 1. level (e.g. info) * 2. source (e.g. javascript.0) * 3. message (e.g. script.js....) * Returns FALSE if no match or input value not valid */ function logLineSplit(inputValue) { // Get RegEx ready let mRegEx = new RegExp(LOG_PATT, 'g'); // Split let returnObj = {} let m; do { m = mRegEx.exec(inputValue); if (m) { returnObj.datetime = m[1]; returnObj.spaceAt2 = m[2]; returnObj.level = m[3]; returnObj.spaceAt4 = m[4]; returnObj.source = m[5]; returnObj.spaceAt6 = m[6]; returnObj.message = m[7]; } } while (m); // Now we check if we have valid entries we want if ((returnObj.datetime === undefined) || (returnObj.level === undefined) || (returnObj.source === undefined) || (returnObj.message === undefined) ) { return false; // no valid hits } // We can return the array now, since it meets all requirements return returnObj; } /** * Merges date/time, level, source and message to a logline * @param {array} inputValue Array with 4 elements: date/time, level, source, message * @return {string} Merged log line as string. Empty string '', if input value not valid. */ function logLineMerge(inputValue) { if (inputValue.length === 4) { let mergedLine = inputValue[0] + ' - ' + inputValue[1] + ': ' + inputValue[2] + ' ' + inputValue[3]; return mergedLine; } else { // We expect a size of 4, so go out return ''; } } /** * Merge Loglines if multiple values and add leading '[123 entries]' to log message * @param {array} logArray array of log entries * @return {array} the new merged log array */ function mergeLogLines(logArray) { // We use array spreads '...' to copy array. If not, array is changed by reference and not value. // That means, if we change the target array, it will also change the source array. // See https://stackoverflow.com/questions/7486085/copy-array-by-value let arrCopy = [...logArray]; let arrNew = []; for (let i = 0; i < arrCopy.length; i++) { if (! isLikeEmpty(arrCopy[i])) { let lpEntry = arrCopy[i]; let lineWithoutDate = lpEntry.substring(23); let lpLineSplit = logLineSplit(lpEntry); // Get multiple values let lpMulti = arrayGetElements(arrCopy, removeLeading123entries(lpLineSplit.message), false); let result = lpEntry; let lineCounter = 0; if (lpMulti.length > 1) { // Treffer - die aktuelle Zeile zählt ja auch mit. lineCounter = lpMulti.length; let hitLeadingNumber = -1; for (let hitLine of lpMulti) { let hitLineSplit = logLineSplit(hitLine); // Check if hit contains '[123 entries]'. If yes, get the number out of it into lineCounter. // If not, we just count with 1. hitLeadingNumber = checkForMultiEntry(hitLineSplit.message); if (hitLeadingNumber > 1) { lineCounter = hitLeadingNumber + lpMulti.length - 1; } } } else { lineCounter = 1; } if (lineCounter > 1) { // remove from array by filling empty value arrCopy = arrayReplaceElementsByValue(arrCopy, removeLeading123entries(lpLineSplit.message), '', false); // new result result = logLineMerge([lpLineSplit.datetime, lpLineSplit.level, lpLineSplit.source, '[' + lineCounter + ' ' + MERGE_LOGLINES_TXT + '] ' + removeLeading123entries(lpLineSplit.message)]); } arrNew.push(result); } } return arrNew; /** * @param {string} strInput A log message with potential leading '[123 entries]' * @return {string} string without leading '[123 entries]', if it is there */ function removeLeading123entries(strInput) { let mRegEx = new RegExp(MERGE_REGEX_PATT); let matches = mRegEx.exec(strInput); if (matches === null) { return strInput; } else { return matches[2]; } } /** * @param {string} strInput A log message checking for leading '[123 entries]' * @return {number} returns the number 123 from '[123 entries]' if any match, or -1 if not found */ function checkForMultiEntry(strInput) { // Get RegEx ready let mRegEx = new RegExp(MERGE_REGEX_PATT); let matches = mRegEx.exec(strInput); if (matches === null) { return -1; } else { return parseInt(matches[1]); } } } /************* * Get the file system path and filename of the current log file. * * ioBroker creates a log file every midnight at 0:00 under '/opt/iobroker/log/' * Syntax of the log file is: iobroker.YYYY-MM-DD.log * This function returns the full path to the log file, considering the current date/time when this function is called. * @return {string} Path and file name to log file. */ function getCurrentFullFsLogPath() { let strLogPathFinal = LOG_FS_PATH; if (strLogPathFinal.slice(-1) !== '/') strLogPathFinal = strLogPathFinal + '/'; let strFullLogPath = strLogPathFinal + DEBUG_CUSTOM_FILENAME; if (DEBUG_CUSTOM_FILENAME === '') strFullLogPath = strLogPathFinal + 'iobroker.' + getCurrentISODate() + '.log'; return strFullLogPath; } /** * Clear array: if stateForTimeStamp is greater or equal than log date, we remove the entire log entry * @param {array} inputArray Array of log entries * @param {string} stateForTimeStamp state of which we need the time stamp * @return {array} cleaned log */ function clearJsonByDate(inputArray, stateForTimeStamp) { let dtState = new Date(getState(stateForTimeStamp).ts); if (SCRIPT_DEBUG) log (DEBUG_IGNORE_STR + 'Time of last change of state [' + stateForTimeStamp + ']: ' + dtState); let newArray = []; for (let lpLog of inputArray) { let dtLog = new Date(lpLog.substr(0,23)); if (dtLog.getTime() >= dtState.getTime()) { newArray.push(lpLog); } } return newArray; } /** * Build an array of states we need to create. * @return {array} 2 array of states to be created. First is for force=false, second is for force=true. */ function buildNeededStates() { let debugCleanIDs = ''; // für Debug-Ausgabe let finalStatesForceFalse = []; // to return all states to be created with option force=false let finalStatesForceTrue = []; // to return all states to be created with option force=true ////////////////////////////////////////////// // All filter states like .logInfo, .logError, etc. ////////////////////////////////////////////// let tmpStatesArray = []; for(let i = 0; i < LOG_FILTER.length; i++) { if (LOG_FILTER[i].id !== '') { let lpIDClean = cleanseStatePath(LOG_FILTER[i].id); debugCleanIDs += ((debugCleanIDs === '') ? '' : '; ') + lpIDClean; // für Debug-Ausgabe tmpStatesArray.push({ id:'log' + lpIDClean + '.log', name:'Filtered Log - ' + lpIDClean, type:"string", role: "state", def: ""}); tmpStatesArray.push({ id:'log' + lpIDClean + '.logJSON', name:'Filtered Log - ' + lpIDClean + ' - JSON', type:"string", role: "state", def: ""}); tmpStatesArray.push({ id:'log' + lpIDClean + '.logJSONcount', name:'Filtered Log - Count of JSON ' + lpIDClean, role: "state", type:"number", def: 0}); tmpStatesArray.push({ id:'log' + lpIDClean + '.clearJSON', name:'Clear JSON log ' + lpIDClean, role: "button", type:"boolean", def: false}); } } // TODO: kind of redundant, just apply all before and push right away into finalStatesForceFalse and not into tempStatesArray for (let s=0; s < tmpStatesArray.length; s++) { finalStatesForceFalse.push([FINAL_STATE_PATH + '.' + tmpStatesArray[s].id, { 'name': tmpStatesArray[s].name, 'desc': tmpStatesArray[s].name, 'type': tmpStatesArray[s].type, 'read': true, 'write': true, 'role': tmpStatesArray[s].role, 'def': tmpStatesArray[s].def, }]); } if (SCRIPT_DEBUG) log (DEBUG_IGNORE_STR + 'createLogStates(): Clean IDs: ' + debugCleanIDs); ////////////////////////////////////////////// // VIS Add-on states (All.visView1, All.visView2, etc.) ////////////////////////////////////////////// if(FINAL_NUM_OF_VIS_VIEWS > 0) { let dropdown = ''; const ALL_FILTER_IDS = getAllFilterIds(); for (let lpEntry of ALL_FILTER_IDS) { dropdown += '"' + lpEntry + '":"' + lpEntry + '",'; // fill JSON string } dropdown = dropdown.substr(0, dropdown.length-1); // remove last comma "," dropdown = '{' + dropdown + '}'; // finalize JSON string let dropdownJSON = JSON.parse(dropdown); // convert to JSON for(let i = 0; i < FINAL_NUM_OF_VIS_VIEWS; i++) { let lpStateVisViews = FINAL_STATE_PATH + '.All.visView' + (i+1); finalStatesForceTrue.push ([lpStateVisViews + '.whichFilter', {'name':'Für VIS-Button/Auswahlmenü des anzuzeigenden Filter in JSON-Tabelle', 'type':'string', 'read':false, 'write':true, 'role':'value', 'states': dropdownJSON, 'def': ALL_FILTER_IDS[0]}]); // force true to rebuild state drop-down finalStatesForceTrue.push ([lpStateVisViews + '.outputJSON', {'name': 'JSON-Ausgabe des in whichFilter gewählten Filters', 'type': 'string', 'read': true, 'write': false, 'role': 'state', 'def': '' }]); // force true to empty on re-start of script. finalStatesForceFalse.push ([lpStateVisViews + '.outputJSONcount', {'name': 'Anzahl Zeilen in JSON-Ausgabe des in whichFilter gewählten Filters', 'type': 'number', 'read': true, 'write': false, 'role': 'state', 'def': 0 }]); finalStatesForceFalse.push([lpStateVisViews + '.clearJSON', {'name':'Clear currently selected JSON Log', 'type':'boolean', 'read':false, 'write':true, 'role':'button', 'def': false }]); } } ////////////////////////////////////////////// // Some more states ////////////////////////////////////////////// // Button zum leeren ALLER JSONs finalStatesForceFalse.push([FINAL_STATE_PATH + '.All.clearAllJSON', {'name':'Clear ALL JSON logs', 'type':'boolean', 'read':false, 'write':true, 'role':'button', 'def': false }]); // Zeit letztes Script-Update (Schedule) finalStatesForceFalse.push([FINAL_STATE_PATH + '.All.lastTimeUpdated', {'name':'Date/Time of last update', 'type':'number', 'read':true, 'write':false, 'role':'value.time'}]); // Done. return [finalStatesForceFalse, finalStatesForceTrue]; } /** * Creates all script states. */ function createLogScriptStates(callback) { const NEEDED_STATES = buildNeededStates(); if (!isLikeEmpty(NEEDED_STATES[0])) { createUserStates(FINAL_STATE_LOCATION, false, NEEDED_STATES[0], function() { forceTrue(); // Next is force=true }); } else { forceTrue(); // Next is force=true } function forceTrue() { if (!isLikeEmpty(NEEDED_STATES[1])) { createUserStates(FINAL_STATE_LOCATION, true, NEEDED_STATES[1], function() { // Done return callback(); }); } else { // Done return callback(); } } } /** * Converts a timestamp to log date format, like 2019-10-15 16:38:00.260. * @param {object} timeStamp The date/time timestamp to convert. * @return {string} The resulting log date format as string. */ function timestampToLogDate(timeStamp) { let date = new Date(timeStamp); // Need to convert to local time as this time provided from onLog() is UTC // https://stackoverflow.com/questions/6525538/convert-utc-date-time-to-local-date-time/18330682 let localDate = new Date(date.getTime() - date.getTimezoneOffset()*60*1000); // Convert to ISO string, so like 2019-10-15T16:38:00.260Z let strResult = localDate.toISOString(); // date.toISOString() adds T and Z, so we remove these letters, as the log do not show these. strResult = strResult.replace('T', ' '); // remove T strResult = strResult.replace('Z', ''); // remove Z at the end return strResult; } /** * Remove PID in log message * The js-controller version 2.0+ adds the PID number inside brackets to the beginning of the message. We remove it here. * @param {string} msg The log message, like: 'javascript.0 (123) Logtext 123 Logtext 123 Logtext 123 Logtext 123' */ function removePID(msg) { // First: Split source and message text. // Input is like: 'javascript.0 (123) Logtext 123 Logtext 123 Logtext 123 Logtext 123' let regexp = /^(\S+)\s(.*)/; let matches_array = msg.match(regexp); let strFirst = matches_array[1]; // like 'javascript.0' let strRest = matches_array[2]; // like '(123) Logtext 123 Logtext 123 Logtext 123 Logtext 123' // Next, we remove the PID strRest = strRest.replace(/^\([0-9]{1,9}\)\s/, ''); // Last, we put the two strings together again return strFirst + ' ' + strRest; } /************************************************************************************************************************* * onStop - Being executed once this ioBroker Script stops. *************************************************************************************************************************/ // This is to end the Tale. Not sure, if we indeed need it, but just in case... onStop(function myScriptStop () { // Unsubscribe log handler onLogUnregister(G_LogHandler); if (LOG_INFO) log('Unsubscribed to Log Handler.', 'info'); }, 0); /** * New in Log-Script 4.7 * Checks a string against a blacklist. * @param {string} stringToCheck String to check against blacklist * @param {array} blacklistArray Array of blacklist. Both strings and regexp are allowed. * If string: it will match if blacklist string is part of provided stringToCheck. * @return {boolean} Returns true, if blacklisted, and false if not. */ function isBlacklisted(stringToCheck, blacklistArray) { if(isLikeEmpty(blacklistArray)) return false; for (let lpBlackItem of blacklistArray) { if(! isLikeEmpty(lpBlackItem)) { if (lpBlackItem instanceof RegExp) { // https://stackoverflow.com/questions/4339288/typeof-for-regexp // We have a regex if ( (stringToCheck.match(lpBlackItem) != null) ) { return true; // We have a hit } } else if (typeof lpBlackItem == 'string') { // No regex, we have a string if(stringToCheck.includes(lpBlackItem)) { return true; // We have a hit } } } } return false; // No hits } /** * New in Log-Script 4.7 * Checks a string against an array of strings or regexp. * @param {string} stringToCheck String to check against array * @param {array} listArray Array of blacklist. Both strings and regexp are allowed. * @param {boolean} all If true, then ALL items of listArray must match to return true. If false, one match or more will return true * @return {boolean} true if matching, false if not. */ function stringMatchesList(stringToCheck, listArray, all) { if(isLikeEmpty(listArray)) return true; let count = 0; let hit = 0; for (let lpListItem of listArray) { if(! isLikeEmpty(lpListItem)) { count = count + 1; if (lpListItem instanceof RegExp) { // https://stackoverflow.com/questions/4339288/typeof-for-regexp // We have a regex if ( (stringToCheck.match(lpListItem) != null) ) { hit = hit + 1; } } else if (typeof lpListItem == 'string') { // No regex, we have a string if(stringToCheck.includes(lpListItem)) { hit = hit + 1; } } } } if (count == 0) return true; if(all) { return (count == hit) ? true : false; } else { return (hit > 0) ? true : false; } } /************************************************************************************************************************* * General supporting functions *************************************************************************************************************************/ /** * Remove Duplicates from Array * Source - https://stackoverflow.com/questions/23237704/nodejs-how-to-remove-duplicates-from-array * @param {array} inputArray Array to process * @return {array} Array without duplicates. */ function arrayRemoveDublicates(inputArray) { let uniqueArray; uniqueArray = inputArray.filter(function(elem, pos) { return inputArray.indexOf(elem) == pos; }); return uniqueArray; } /** * Clean Array: Removes all falsy values: undefined, null, 0, false, NaN and "" (empty string) * Source: https://stackoverflow.com/questions/281264/remove-empty-elements-from-an-array-in-javascript * @param {array} inputArray Array to process * @return {array} Cleaned array */ function cleanArray(inputArray) { var newArray = []; for (let i = 0; i < inputArray.length; i++) { if (inputArray[i]) { newArray.push(inputArray[i]); } } return newArray; } /** * Checks if Array or String is not undefined, null or empty. * 08-Sep-2019: added check for [ and ] to also catch arrays with empty strings. * @param inputVar - Input Array or String, Number, etc. * @return true if it is undefined/null/empty, false if it contains value(s) * Array or String containing just whitespaces or >'< or >"< or >[< or >]< is considered empty */ function isLikeEmpty(inputVar) { if (typeof inputVar !== 'undefined' && inputVar !== null) { let strTemp = JSON.stringify(inputVar); strTemp = strTemp.replace(/\s+/g, ''); // remove all whitespaces strTemp = strTemp.replace(/\"+/g, ""); // remove all >"< strTemp = strTemp.replace(/\'+/g, ""); // remove all >'< strTemp = strTemp.replace(/[+/g, ""); // remove all >[< strTemp = strTemp.replace(/]+/g, ""); // remove all >]< if (strTemp !== '') { return false; } else { return true; } } else { return true; } } /** * Returns the current date in ISO format "YYYY-MM-DD". * @return {string} Date in ISO format */ function getCurrentISODate() { let currDate = new Date(); return currDate.getFullYear() + '-' + zeroPad((currDate.getMonth() + 1), 2) + '-' + zeroPad(currDate.getDate(), 2); } /** * Fügt Vornullen zu einer Zahl hinzu, macht also z.B. aus 7 eine "007". * zeroPad(5, 4); // wird "0005" * zeroPad('5', 6); // wird "000005" * zeroPad(1234, 2); // wird "1234" :) * @param {string|number} num Zahl, die Vornull(en) bekommen soll * @param {number} places Anzahl Stellen. * @return {string} Zahl mit Vornullen wie gewünscht. */ function zeroPad(num, places) { let zero = places - num.toString().length + 1; return Array(+(zero > 0 && zero)).join("0") + num; } /** * Will just keep lower case letters, numbers, '-' and '_' and removes the rest * Also, capitalize first Letter. */ function cleanseStatePath(stringInput) { let strProcess = stringInput; strProcess = strProcess.replace(/([^a-z0-9_\-]+)/gi, ''); strProcess = strProcess.toLowerCase(); strProcess = strProcess.charAt(0).toUpperCase() + strProcess.slice(1); return strProcess; } /** * Checks if a a given state or part of state is existing. * This is a workaround, as getObject() or getState() throw warnings in the log. * Set strict to true if the state shall match exactly. If it is false, it will add a wildcard * to the end. * See: https://forum.iobroker.net/topic/11354/ * @param {string} strStatePath Input string of state, like 'javas-cript.0.switches.Osram.Bedroom' * @param {boolean} [strict=true] Optional: Default is true. If true, it will work strict, if false, it will add a wildcard * to the end of the string * @return {boolean} true if state exists, false if not */ function isState(strStatePath, strict) { if(strict === undefined) strict = true; let mSelector; if (strict) { mSelector = $('state[id=' + strStatePath + '$]'); } else { mSelector = $('state[id=' + strStatePath + ']'); } if (mSelector.length > 0) { return true; } else { return false; } } /** * Removing Array element(s) by input value. * @param {array} arr the input array * @param {string} valRemove the value to be removed * @param {boolean} [exact=true] OPTIONAL: default is true. if true, it must fully match. if false, it matches also if valRemove is part of element string * @return {array} the array without the element(s) */ function arrayRemoveElementsByValue(arr, valRemove, exact) { for ( let i = 0; i < arr.length; i++){ if (exact) { if ( arr[i] === valRemove) { arr.splice(i, 1); i--; // required, see https://love2dev.com/blog/javascript-remove-from-array/ } } else { if (arr[i].indexOf(valRemove) != -1) { arr.splice(i, 1); i--; // see above } } } return arr; } /** * Escapes a string for use in RegEx as (part of) pattern * Source: https://stackoverflow.com/questions/3446170/escape-string-for-use-in-javascript-regex * @param {string} inputStr The input string to be escaped * @return {string} The escaped string */ function escapeRegExp(inputStr) { return inputStr.replace(/[.*+?^${}()|[]\]/g, '\\$&'); // $& means the whole matched string } /** * Get all elements of an array if found * @param {array} arr the input array * @param {string} valFind the value to find * @param {boolean} [exact=true] OPTIONAL: default is true. if true, it must fully match. if false, it matches also if valRemove is part of element string * @return {array} an array with all hits or empty array if no hits. */ function arrayGetElements(arr, valFind, exact) { let resultArr = []; for ( let i = 0; i < arr.length; i++){ if (exact) { if ( arr[i] === valFind) { resultArr.push(arr[i]); } } else { if (arr[i].indexOf(valFind) != -1) { resultArr.push(arr[i]); } } } return resultArr; } /** * Replace Array element(s) by input value. * @param {array} arr the input array * @param {string} valReplace the value to search for * @param {string} newValue the new value * @param {boolean} [exact=true] OPTIONAL: default is true. if true, it must fully match. if false, it matches also if valRemove is part of element string * @return {array} the array with replaced the element(s) */ function arrayReplaceElementsByValue(arr, valReplace, newValue, exact) { for ( let i = 0; i < arr.length; i++){ if (exact) { if ( arr[i] === valReplace) { arr[i] = newValue; } } else { if (arr[i].indexOf(valReplace) != -1) { arr[i] = newValue; } } } return arr; } /** * Retrieve values from a CONFIG variable, example: * const CONF = [{car: 'bmw', color: 'black', hp: '250'}, {car: 'audi', color: 'blue', hp: '190'}] * To get the color of the Audi, use: getConfigValuePerKey(CONF, 'car', 'audi', 'color') * To find out which car has 190 hp, use: getConfigValuePerKey(CONF, 'hp', '190', 'car') * @param {object} config The configuration variable/constant * @param {string} key1 Key to look for. * @param {string} key1Value The value the key should have * @param {string} key2 The key which value we return * @returns {any} Returns the element's value, or number -1 of nothing found. */ function getConfigValuePerKey(config, key1, key1Value, key2) { for (let lpConfDevice of config) { if ( lpConfDevice[key1] === key1Value ) { if (lpConfDevice[key2] === undefined) { return -1; } else { return lpConfDevice[key2]; } } } return -1; } /** * For a given state path, we extract the location '0_userdata.0' or 'javascript.0' or add '0_userdata.0', if missing. * @param {string} path Like: 'Computer.Control-PC', 'javascript.0.Computer.Control-PC', '0_userdata.0.Computer.Control-PC' * @param {boolean} returnFullPath If true: full path like '0_userdata.0.Computer.Control-PC', if false: just location like '0_userdata.0' or 'javascript.0' * @return {string} Path */ function validateStatePath(path, returnFullPath) { if (path.startsWith('.')) path = path.substr(1); // Remove first dot if (path.endsWith('.')) path = path.slice(0, -1); // Remove trailing dot if (path.length < 1) log('Provided state path is not valid / too short.', 'error') let match = path.match(/^((javascript\.([1-9][0-9]|[0-9])\.)|0_userdata\.0\.)/); let location = (match == null) ? '0_userdata.0' : match[0].slice(0, -1); // default is '0_userdata.0'. if(returnFullPath) { return (path.indexOf(location) == 0) ? path : (location + '.' + path); } else { return location; } } /** * Create states under 0_userdata.0 or javascript.x * Current Version: https://github.com/Mic-M/iobroker.createUserStates * Support: https://forum.iobroker.net/topic/26839/ * Autor: Mic (ioBroker) | Mic-M (github) * Version: 1.1 (26 January 2020) * Example: see https://github.com/Mic-M/iobroker.createUserStates#beispiel * ----------------------------------------------- * PLEASE NOTE: Per https://github.com/ioBroker/ioBroker.javascript/issues/474, the used function setObject() * executes the callback PRIOR to completing the state creation. Therefore, we use a setTimeout and counter. * ----------------------------------------------- * @param {string} where Where to create the state: '0_userdata.0' or 'javascript.x'. * @param {boolean} force Force state creation (overwrite), if state is existing. * @param {array} statesToCreate State(s) to create. single array or array of arrays * @param {object} [callback] Optional: a callback function -- This provided function will be executed after all states are created. */ function createUserStates(where, force, statesToCreate, callback = undefined) { const WARN = false; // Only for 0_userdata.0: Throws warning in log, if state is already existing and force=false. Default is false, so no warning in log, if state exists. const LOG_DEBUG = false; // To debug this function, set to true // Per issue #474 (https://github.com/ioBroker/ioBroker.javascript/issues/474), the used function setObject() executes the callback // before the state is actual created. Therefore, we use a setTimeout and counter as a workaround. const DELAY = 50; // Delay in milliseconds (ms). Increase this to 100, if it is not working. // Validate "where" if (where.endsWith('.')) where = where.slice(0, -1); // Remove trailing dot if ( (where.match(/^((javascript\.([1-9][0-9]|[0-9]))$|0_userdata\.0$)/) == null) ) { log('This script does not support to create states under [' + where + ']', 'error'); return; } // Prepare "statesToCreate" since we also allow a single state to create if(!Array.isArray(statesToCreate[0])) statesToCreate = [statesToCreate]; // wrap into array, if just one array and not inside an array // Add "where" to STATES_TO_CREATE for (let i = 0; i < statesToCreate.length; i++) { let lpPath = statesToCreate[i][0].replace(/\.*\./g, '.'); // replace all multiple dots like '..', '...' with a single '.' lpPath = lpPath.replace(/^((javascript\.([1-9][0-9]|[0-9])\.)|0_userdata\.0\.)/,'') // remove any javascript.x. / 0_userdata.0. from beginning lpPath = where + '.' + lpPath; // add where to beginning of string statesToCreate[i][0] = lpPath; } if (where != '0_userdata.0') { // Create States under javascript.x let numStates = statesToCreate.length; statesToCreate.forEach(function(loopParam) { if (LOG_DEBUG) log('[Debug] Now we are creating new state [' + loopParam[0] + ']'); let loopInit = (loopParam[1]['def'] == undefined) ? null : loopParam[1]['def']; // mimic same behavior as createState if no init value is provided createState(loopParam[0], loopInit, force, loopParam[1], function() { numStates--; if (numStates === 0) { if (LOG_DEBUG) log('[Debug] All states processed.'); if (typeof callback === 'function') { // execute if a function was provided to parameter callback if (LOG_DEBUG) log('[Debug] Function to callback parameter was provided'); return callback(); } else { return; } } }); }); } else { // Create States under 0_userdata.0 let numStates = statesToCreate.length; let counter = -1; statesToCreate.forEach(function(loopParam) { counter += 1; if (LOG_DEBUG) log ('[Debug] Currently processing following state: [' + loopParam[0] + ']'); if( ($(loopParam[0]).length > 0) && (existsState(loopParam[0])) ) { // Workaround due to https://github.com/ioBroker/ioBroker.javascript/issues/478 // State is existing. if (WARN && !force) log('State [' + loopParam[0] + '] is already existing and will no longer be created.', 'warn'); if (!WARN && LOG_DEBUG) log('[Debug] State [' + loopParam[0] + '] is already existing. Option force (=overwrite) is set to [' + force + '].'); if(!force) { // State exists and shall not be overwritten since force=false // So, we do not proceed. numStates--; if (numStates === 0) { if (LOG_DEBUG) log('[Debug] All states successfully processed!'); if (typeof callback === 'function') { // execute if a function was provided to parameter callback if (LOG_DEBUG) log('[Debug] An optional callback function was provided, which we are going to execute now.'); return callback(); } } else { // We need to go out and continue with next element in loop. return; // https://stackoverflow.com/questions/18452920/continue-in-cursor-foreach } } // if(!force) } // State is not existing or force = true, so we are continuing to create the state through setObject(). let obj = {}; obj.type = 'state'; obj.native = {}; obj.common = loopParam[1]; setObject(loopParam[0], obj, function (err) { if (err) { log('Cannot write object for state [' + loopParam[0] + ']: ' + err); } else { if (LOG_DEBUG) log('[Debug] Now we are creating new state [' + loopParam[0] + ']') let init = null; if(loopParam[1].def === undefined) { if(loopParam[1].type === 'number') init = 0; if(loopParam[1].type === 'boolean') init = false; if(loopParam[1].type === 'string') init = ''; } else { init = loopParam[1].def; } setTimeout(function() { setState(loopParam[0], init, true, function() { if (LOG_DEBUG) log('[Debug] setState durchgeführt: ' + loopParam[0]); numStates--; if (numStates === 0) { if (LOG_DEBUG) log('[Debug] All states processed.'); if (typeof callback === 'function') { // execute if a function was provided to parameter callback if (LOG_DEBUG) log('[Debug] Function to callback parameter was provided'); return callback(); } } }); }, DELAY + (20 * counter) ); } }); }); } }
