Mit Home Assistant Webseiten-Daten auslesen & in Entitäten sichern

Wenn du Daten von Webseiten mithilfe von Home Assistant auslesen und in Entitäten speichern willst, bist du hier genau richtig. Ich zeige dir, wie du für den Scrape-Sensor den passenden Selektor findest und des Weiteren eine alternative Möglichkeit für komplexe Webseiten über Node-RED.

Das Ganze erkläre ich am Beispiel der Web-Oberfläche meines IBC-Solar Servemaster Plus Wechselrichters (Danfoss) 🙌

Inhaltsverzeichnis

• Empfohlenes Vorwissen
• Genutzt Integration
• Konfigurierte Scrape-Sensoren
  • Beispiele mit der Home Assistant Seite
    • Aktuelle Version
    • Test aus dem Video
  • Beispiel für den Wechselrichter
• Node-RED Flow zum Auslesen des Wechselrichters
• Fazit

Empfohlenes Vorwissen

• Home Assistant Template Sensoren & Hilfsentitäten (Helfer) 101

Genutzt Integration

• https://www.home-assistant.io/integrations/scrape/

Konfigurierte Scrape-Sensoren

Beispiele mit der Home Assistant Seite

Aktuelle Version

scrape:
  - resource: https://www.home-assistant.io
    sensor:
      - name: "Current version"
        select: ".current-version h1"

Test aus dem Video

scrape:
  - resource: https://www.home-assistant.io
    sensor:
      - name: "Join The Community Sensor"
        select: "body > div.page-content > div.grid-wrapper > div > div > article > div > div > div.grid__item.one-third.lap-one-third.palm-one-whole > div.join-community.material-card.text > h1"
  - resource: https://www.home-assistant.io
    sensor:
      - name: "Alternative für Join The Community Sensor"
        select: ".join-community h1"

Beispiel für den Wechselrichter

scrape:
  - resource: http://192.168.20.67/cgi-bin/overview.tcl?sid=4618340214069137192&menuParentId=3
    authentication: basic
    username: admin
    password: admin
    scan_interval: 10
    sensor:
      - name: Aktuelle Leistung in Watt SI
        select: "#curr_power"
        value_template: '{{ value.split(" ")[0] }}'
        unit_of_measurement: 'W'
        device_class: "power"
        state_class: "measurement"

Node-RED Flow zum Auslesen des Wechselrichters

[{"id":"93eabd9642fa1cd8","type":"tab","label":"Get Solar Power Values","disabled":false,"info":"","env":[]},{"id":"783cae8c9aa1054c","type":"http request","z":"93eabd9642fa1cd8","name":"Get Session ID","method":"POST","ret":"txt","paytoqs":"ignore","url":"http://192.168.20.67/cgi-bin/handle_login.tcl","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[{"keyType":"other","keyValue":"Content-Type","valueType":"other","valueValue":"multipart/form-data"},{"keyType":"other","keyValue":"Accept","valueType":"other","valueValue":"*/*"},{"keyType":"other","keyValue":"Accept-Encoding","valueType":"other","valueValue":"gzip, deflate, br"}],"x":1180,"y":540,"wires":[["4f4dd65560a20597","b0da5025fc2d3c9c"]]},{"id":"0c757c80cca49353","type":"function","z":"93eabd9642fa1cd8","name":"Set form-data via fnc","func":"msg.payload = {\n    \"user\": \"admin\",\n    \"pw\": \"admin\",\n    \"submit\": \"Anmelden\"\n};\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":960,"y":540,"wires":[["783cae8c9aa1054c"]]},{"id":"e11e51d067a9acf7","type":"api-call-service","z":"93eabd9642fa1cd8","name":"Set SID Helper","server":"c520191b.82f468","version":5,"debugenabled":true,"domain":"input_text","service":"set_value","areaId":[],"deviceId":[],"entityId":["input_text.solaranlage_session_sid_text"],"data":"{\"value\": \"\" & msg.ddNewSID & \"\"}","dataType":"jsonata","mergeContext":"","mustacheAltTags":false,"outputProperties":[],"queue":"none","x":1600,"y":540,"wires":[[]]},{"id":"3f815e9ef1f042f7","type":"inject","z":"93eabd9642fa1cd8","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"20","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":110,"y":380,"wires":[["f4f5170e0fb22860"]]},{"id":"429bdfa84b96d42b","type":"http request","z":"93eabd9642fa1cd8","name":"Get Website Data","method":"GET","ret":"txt","paytoqs":"ignore","url":"","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"basic","senderr":false,"headers":[],"x":130,"y":540,"wires":[["4139277f3680d37f"]]},{"id":"6d4c20806cd4d4c4","type":"ha-get-entities","z":"93eabd9642fa1cd8","name":"","server":"c520191b.82f468","version":0,"rules":[{"property":"","logic":"is","value":"","valueType":"str"}],"output_type":"array","output_empty_results":false,"output_location_type":"msg","output_location":"CurrentSessionID","output_results_count":1,"x":90,"y":940,"wires":[[]]},{"id":"f4f5170e0fb22860","type":"api-render-template","z":"93eabd9642fa1cd8","name":"Get SID","server":"c520191b.82f468","version":0,"template":"{{ states('input_text.solaranlage_session_sid_text') }}","resultsLocation":"CurrentSIDFromTemplate","resultsLocationType":"msg","templateLocation":"template","templateLocationType":"msg","x":260,"y":380,"wires":[["ce4517086692314f"]]},{"id":"d8fafe7ea29f8eb8","type":"debug","z":"93eabd9642fa1cd8","name":"total_yield","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"total_yield","targetType":"msg","statusVal":"","statusType":"auto","x":570,"y":740,"wires":[]},{"id":"316b7ad651661c6e","type":"debug","z":"93eabd9642fa1cd8","name":"curr_power","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"curr_power","targetType":"msg","statusVal":"","statusType":"auto","x":310,"y":740,"wires":[]},{"id":"ce4517086692314f","type":"function","z":"93eabd9642fa1cd8","name":"Set form-data via fnc","func":"msg.payload = {\n};\nmsg.url = \"http://192.168.20.67/cgi-bin/overview.tcl?sid=\" + msg.CurrentSIDFromTemplate + \"&menuParentId=3\"\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":480,"y":380,"wires":[["429bdfa84b96d42b"]]},{"id":"4139277f3680d37f","type":"switch","z":"93eabd9642fa1cd8","name":"","property":"payload","propertyType":"msg","rules":[{"t":"cont","v":"Die Webschnittstelle wird momentan anderweitig","vt":"str"},{"t":"cont","v":"Wechselrichterstatus","vt":"str"},{"t":"else"}],"checkall":"true","repair":false,"outputs":3,"x":370,"y":540,"wires":[["9d8a11cde92e2665"],["16e89a62f7d046b3"],["9d8a11cde92e2665"]]},{"id":"16e89a62f7d046b3","type":"html","z":"93eabd9642fa1cd8","name":"Get curr_power","property":"payload","outproperty":"curr_power","tag":"#curr_power","ret":"text","as":"multi","x":140,"y":680,"wires":[["316b7ad651661c6e","58916ff0bafbae3c","e424a601110803e8"]]},{"id":"e424a601110803e8","type":"html","z":"93eabd9642fa1cd8","name":"Get total_yield","property":"payload","outproperty":"total_yield","tag":"#total_yield","ret":"text","as":"multi","x":400,"y":680,"wires":[["08a44f7c0bde0989","d8fafe7ea29f8eb8","12b9ad486ba22fe8"]]},{"id":"08a44f7c0bde0989","type":"html","z":"93eabd9642fa1cd8","name":"Get prod_today","property":"payload","outproperty":"prod_today","tag":"#prod_today","ret":"text","as":"multi","x":640,"y":680,"wires":[["ec7471c0cab9f9e9","50a14453a1f889a1"]]},{"id":"ec7471c0cab9f9e9","type":"debug","z":"93eabd9642fa1cd8","name":"prod_today","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"prod_today","targetType":"msg","statusVal":"","statusType":"auto","x":810,"y":740,"wires":[]},{"id":"58916ff0bafbae3c","type":"api-call-service","z":"93eabd9642fa1cd8","name":"Set Raw Value","server":"c520191b.82f468","version":5,"debugenabled":false,"domain":"input_text","service":"set_value","areaId":[],"deviceId":[],"entityId":["input_text.solaranlage_curr_power_raw"],"data":"{\"value\": \"\" & msg.curr_power & \"\"}","dataType":"jsonata","mergeContext":"","mustacheAltTags":false,"outputProperties":[],"queue":"none","x":300,"y":820,"wires":[[]]},{"id":"12b9ad486ba22fe8","type":"api-call-service","z":"93eabd9642fa1cd8","name":"Set Raw Value","server":"c520191b.82f468","version":5,"debugenabled":false,"domain":"input_text","service":"set_value","areaId":[],"deviceId":[],"entityId":["input_text.solaranlage_total_yield_raw"],"data":"{\"value\": \"\" & msg.total_yield & \"\"}","dataType":"jsonata","mergeContext":"","mustacheAltTags":false,"outputProperties":[],"queue":"none","x":580,"y":800,"wires":[[]]},{"id":"50a14453a1f889a1","type":"api-call-service","z":"93eabd9642fa1cd8","name":"Set Raw Value","server":"c520191b.82f468","version":5,"debugenabled":false,"domain":"input_text","service":"set_value","areaId":[],"deviceId":[],"entityId":["input_text.solaranlage_prod_today_raw"],"data":"{\"value\": \"\" & msg.prod_today & \"\"}","dataType":"jsonata","mergeContext":"","mustacheAltTags":false,"outputProperties":[],"queue":"none","x":820,"y":800,"wires":[[]]},{"id":"4f4dd65560a20597","type":"debug","z":"93eabd9642fa1cd8","name":"Web-Response","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":1360,"y":600,"wires":[]},{"id":"b0da5025fc2d3c9c","type":"function","z":"93eabd9642fa1cd8","name":"Extract SID","func":"msg.payload = msg.payload.split(\"?sid=\")[1]\nmsg.payload = msg.payload.split(\"'\\\">Anm\")[0]\nmsg.ddNewSID = msg.payload\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1390,"y":540,"wires":[["ae141368bd20230a","e11e51d067a9acf7"]]},{"id":"ae141368bd20230a","type":"debug","z":"93eabd9642fa1cd8","name":"New SID","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"ddNewSID","targetType":"msg","statusVal":"","statusType":"auto","x":1560,"y":600,"wires":[]},{"id":"f200918891981606","type":"switch","z":"93eabd9642fa1cd8","name":"Is 10 min?","property":"curminutes","propertyType":"msg","rules":[{"t":"eq","v":"11","vt":"str"},{"t":"eq","v":"1","vt":"str"},{"t":"eq","v":"21","vt":"str"},{"t":"eq","v":"31","vt":"str"},{"t":"eq","v":"41","vt":"str"},{"t":"eq","v":"51","vt":"str"},{"t":"else"}],"checkall":"true","repair":false,"outputs":7,"x":750,"y":540,"wires":[["0c757c80cca49353"],["0c757c80cca49353"],["0c757c80cca49353"],["0c757c80cca49353"],["0c757c80cca49353"],["0c757c80cca49353"],["4bf2db22f9ebbd2f"]]},{"id":"9d8a11cde92e2665","type":"function","z":"93eabd9642fa1cd8","name":"Get Minutes","func":"var time = new Date();\nmsg.curminutes = time.getMinutes();\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":570,"y":540,"wires":[["f200918891981606"]]},{"id":"4bf2db22f9ebbd2f","type":"delay","z":"93eabd9642fa1cd8","name":"","pauseType":"delay","timeout":"60","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":950,"y":620,"wires":[[]]},{"id":"c520191b.82f468","type":"server","name":"Home Assistant","addon":true}]

Zugehörige Template-Sensoren

template:
  - sensor:
    # Errechnet aus den Rohdaten den kWh Wert
      - name: Ertrag heute Solaranlage (kWh) Total Inc
        unique_id: "ErtragHeuteSolaranlagekWh"
        unit_of_measurement: 'kWh'
        device_class: "energy"
        state_class: "total_increasing"
        state: >-
            {% if 'kWh' in states('input_text.solaranlage_prod_today_raw') %}
            {{ states('input_text.solaranlage_prod_today_raw').split(" ")[0] | float }}
            {% else %} 
            {{ states('input_text.solaranlage_prod_today_raw').split(" ")[0] | float / 1000 }}
            {% endif %}
  - sensor:
    # Errechnet aus den Rohdaten den MWh Wert
      - name: Gesamtertrag Solaranlage
        unique_id: "GesamtertragSolaranlageMWh"
        unit_of_measurement: 'MWh'
        device_class: "energy"
        state_class: "total"
        state: >-
            {% if 'kWh' in states('input_text.solaranlage_total_yield_raw') %}
            {{ states('input_text.solaranlage_total_yield_raw').split(" ")[0] | float / 1000 }}
            {% else %} 
            {{ states('input_text.solaranlage_total_yield_raw').split(" ")[0] | float }}
            {% endif %}
  - sensor:
    # Errechnet aus den Rohdaten den Watt Wert
      - name: Aktuelle Leistung Solaranlage
        unique_id: "AktuelleLeistungSolaranlageWatt"
        unit_of_measurement: 'W'
        device_class: "power"
        state_class: "measurement"
        state: >-
            {% if 'Kilo' in states('input_text.solaranlage_curr_power_raw') %}
            {{ states('input_text.solaranlage_curr_power_raw').split(" ")[0] | float / 1000 }}
            {% else %} 
            {{ states('input_text.solaranlage_curr_power_raw').split(" ")[0] | float }}
            {% endif %}

Fazit

Ich hoffe die hat mein Video zum Auslesen von Webseiten mit Home Assistant und den Scrape Sensoren gefallen, und du kannst nun auch nach Belieben Entitäten erstellen, die Werte von Webseiten beinhalten.

Natürlich kannst du, wenn du ebenfalls so einen IBC Solar Wechselrichter hast, den Code einfach kopieren… Aber ich denke mal, das ist bei den wenigsten der Fall 😅

Falls du Fragen oder Anregungen hast, zögere bitte nicht, mir ein Kommentar zu schreiben.

Bitte dieses Feld leer lassen

smarte News

Erfahre als Erster, wenn wir neue Artikel zu Smart Home oder anderen interessanten Themen veröffentlichen!

E-Mail-Adresse *

Wir senden keinen Spam und verkaufen keine Daten! Erfahre mehr in unserer Datenschutzerklärung.

Prüfe deinen Posteingang oder Spam-Ordner, um dein Abonnement zu bestätigen.