Wojtecs homepage

Monitorowanie otoczenia

Napisał

Wojciech Kawecki

in

Do monitorowania obrazu z kamera do niedawna używałem addona motionEye, jednak ostatnio przesiadłem się na addon Frigate. Oferuje znacznie większe możliwości, takie jak rozpoznawanie określonych obiektów oraz dzięków. W tym rozwiązaniu jest używany akcelerator NPU Hailo8L, odciża procesor RP5 z operacji analizy obrazu. Hailo ma bardzo dobry współczynnik ilości obliczeń do zużywanej mocy. Całość jest zasilana ze standardowego zasilacza RPI5.

Hardware

  • „tania” chińska kamera P05-7
  • ESP32 + kamera OV2460
  • Raspberry Pi AI Kit (M.2 HAT+ i akcelerator Hailo-8L) – link
  • (opcjonalnie) TV z GoogleTV
  • (opcjonalnie) Telefon z Androidem wraz ze skonfigurowaną aplikacją HA

Software

  • Addon: Frigate HailoRT
  • Integracja HACS: Frigate Home Assistant Integration
  • Komponent HACS: Advanced Camera Card
  • (opcjonalnie) TvOverlay – aplikacja umożliwiająca wyświetlanie powiadomień na Google TV
  • uruchomiony serwer Samba

Konfiguracja HA

Używam podłączonego dysku USB do przechowywania nagrań. Dysk jest skonfigurowany jako dysk sieciowy przy użyciu Samby.

W HA: Ustawienia -> System -> Pamięć masowa

Instalacja Hailo

Podłączenie Hailo do RPi5 – link

Konfiguracja systemu pod Hailo – link

Niestety paczka hailo-all zawiera firmware i sterownik w wersji 4.20, a wersja Frigate HailoRT 0.15 zwiera oprogramowanie do Hailo w wersji 4.19. Konieczne jest przygotowanie własnej wersji sterownika i frimware do obsługi Hailo. Szczegóły rozwiązania w artykule link.

Konfiguracja Frigate

mqtt:
  host: 192.168.1.2
  user: MQTT_USER
  password: MQTT_PASSWORD

ffmpeg:
  hwaccel_args: preset-rpi-64-h264

record:
  enabled: true
  retain:
    days: 3
    mode: motion
  alerts:
    retain:
      days: 10
      mode: motion
  detections:
    retain:
      days: 10
      mode: motion
timestamp_style:
  position: bl
  format: '%Y.%m.%d %H:%M:%S'

detectors:
  hailo8l:
    type: hailo8l
    device: PCIe
#    model:
#      # from https://github.com/hailo-ai/hailo_model_zoo/blob/master/docs/public_models/HAILO8L/HAILO8L_object_detection.rst

model:
  width: 640
  height: 640
  input_tensor: nhwc
  input_pixel_format: bgr
  model_type: yolox
  path: /config/model_cache/h8l_cache/yolov8s.hef
  labelmap:
    0: person
    1: bicycle
    2: car
    3: motorcycle
    4: airplane
    5: bus
    6: train
    7: truck
    8: boat
    9: traffic light
    10: fire hydrant
    11: stop sign
    12: parking meter
    13: bench
    14: bird
    15: cat
    16: dog
    17: horse
    18: sheep
    19: cow
    20: elephant
    21: bear
    22: zebra
    23: giraffe
    24: backpack
    25: umbrella
    26: handbag
    27: tie
    28: suitcase
    29: frisbee
    30: skis
    31: snowboard
    32: sports ball
    33: kite
    34: baseball bat
    35: baseball glove
    36: skateboard
    37: surfboard
    38: tennis racket
    39: bottle
    40: wine glass
    41: cup
    42: fork
    43: knife
    44: spoon
    45: bowl
    46: banana
    47: apple
    48: sandwich
    49: orange
    50: broccoli
    51: carrot
    52: hot dog
    53: pizza
    54: donut
    55: cake
    56: chair
    57: couch
    58: potted plant
    59: bed
    60: dining table
    61: toilet
    62: TV
    63: laptop
    64: mouse
    65: remote
    66: keyboard
    67: cell phone
    68: microwave
    69: oven
    70: toaster
    71: sink
    72: refrigerator
    73: book
    74: clock
    75: vase
    76: scissors
    77: teddy bear
    78: hair drier
    79: toothbrush

#detectors:
#  hailo8l:
#    type: hailo8l
#    device: PCIe

#model:
#  width: 300
#  height: 300
#  input_tensor: nhwc
#  input_pixel_format: bgr
#  model_type: ssd
#  path: /config/model_cache/h8l_cache/ssd_mobilenet_v1.hef

#objects:
#  track:
#    - person
#    - car
#    - face
#    - license_plate
#    - dog
#    - cat
#    - package
#  filters:
#    person:
#      threshold: 0.40
#    car:
#      threshold: 0.40
#    face:
#      threshold: 0.40
#    license_plate:
#      threshold: 0.40
#    dog:
#      threshold: 0.40
#    cat:
#      threshold: 0.40
#    package:
#      threshold: 0.40

snapshots:
  enabled: true
  timestamp: true
  bounding_box: true
  retain:
    default: 14

go2rtc:
  streams:
    Wejscie2: # <- for RTSP streams
      - rtsp://192.168.1.137:8001/ # <- stream which supports video & aac audio

cameras:
  Wejscie2: # <------ Name the camera
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Wejscie2 # <----- The stream you want to use for detection
          roles:
            - detect
            - record
    record:
      enabled: true
    detect:
      enabled: true # <---- disable detection until you have a working camera feed
#      width: 1280
#      height: 720
#      fps: 15
    mqtt:
      enabled: true
      timestamp: true
      bounding_box: true
      crop: true
      quality: 90
      required_zones:
        - Chodnik
    motion:
      threshold: 30
      mask: 0,0.058,0.254,0.057,0.255,0.012,0.003,0.01
      contour_area: 20
      improve_contrast: true
#    onvif:
#      host: 192.168.4.137
#      port: 8000
#      user: admin
#      ignore_time_mismatch: true

    zones:
      Ulica:
        coordinates: 0.532,0.276,0.658,0.321,0.909,0.462,1,0.509,0.999,0.003,0.531,0.004
        loitering_time: 0
        inertia: 3
      Chodnik:
        coordinates: 0.325,1,0.842,0.428,0.65,0.324,0.534,0.284,0.532,0.134,0.003,0.262,0,1
        loitering_time: 0
        inertia: 3
      Parking:
        coordinates: 0.847,0.433,1,0.51,0.998,0.998,0.332,1
        loitering_time: 0
    review:
      alerts:
        required_zones:
          - Ulica
          - Chodnik
      detections:
        required_zones:
          - Ulica
          - Chodnik
  Garaz2:
    enabled: true
    ffmpeg:
      inputs:
        - path: http://192.168.1.138:8080/
          roles:
            - detect
            - record
      input_args: -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict
        experimental -fflags +genpts+discardcorrupt -use_wallclock_as_timestamps 1
        -c:v mjpeg
      output_args:
        detect: -f rawvideo -pix_fmt yuv420p -an
#        record: -f segment -pix_fmt yuv420p -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v libx264 -preset ultrafast -an
        record: -f segment -pix_fmt yuv420p -segment_time 10 -segment_format mp4 -reset_timestamps
          1 -strftime 1 -preset ultrafast -an
#        rtmp: -f flv -pix_fmt yuv420p -c:v libx264 -preset ultrafast -an
    record:
      enabled: true
    detect:
      enabled: true
      width: 800
      height: 600
      fps: 5
#    objects:
#      track:
#        - person
#        - face
#        - package
#        - cup
    mqtt:
      enabled: true
      timestamp: true
      bounding_box: true
      crop: false
      quality: 100
      height: 600
    motion:
      threshold: 35
      contour_area: 10
      improve_contrast: false
  Podjazd:
    enabled: false
    ffmpeg:
      inputs:
        - path: http://192.168.1.139:8080/
          roles:
            - detect
            - record
      input_args: -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict
        experimental -fflags +genpts+discardcorrupt -use_wallclock_as_timestamps 1
        -c:v mjpeg
      output_args:
        detect: -f rawvideo -pix_fmt yuv420p -an
#        record: -f segment -pix_fmt yuv420p -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v libx264 -preset ultrafast -an
        record: -f segment -pix_fmt yuv420p -segment_time 10 -segment_format mp4 -reset_timestamps
          1 -strftime 1 -preset ultrafast -an
#        rtmp: -f flv -pix_fmt yuv420p -c:v libx264 -preset ultrafast -an
    record:
      enabled: true
    detect:
      enabled: true
      fps: 5
#    objects:
#      track:
#        - person
#        - face
#        - package
#        - cup
    mqtt:
      enabled: true
      timestamp: true
      bounding_box: true
      crop: false
      quality: 100
      height: 600
    motion:
      threshold: 35
      contour_area: 10
      improve_contrast: false
version: 0.15-1

Ja korzystam z modelu yolov8s, który należy wcześniej wgrać do katalogu homeassistant/model_cache/h8l_cache

Powiadomienia na telewizorze GoogleTV (opcjonalnie)

Telewizor z GoogleTV z aplikacją TVOverlay oraz uruchomiony serwer MQTT na instancji HA.

Konfiguracja TvOverlay

Uruchamiamy aplikację na TV i ustawiamy:

  • Background Visibility na 0%
  • Clock Visibility na 0%
  • MQTT – IP, login i hasło serwera MQTT
  • More settings -> Battery Optimalization

Konfigurację można zweryfikować pobierając XMLa z linku http://IP_TV:5001

Konfiguracja TvOverlay w HA

Należy dodać w configuration.yaml w sekcji notify poniższy kod (ustawiając IP_TV i port – domyślnie 5001)

notify:
  - name: TvOverlayNotify
    platform: rest
    method: POST_JSON
    resource: http://[ip]:[port]/notify 
    verify_ssl: false
    title_param_name: title
    data:
      id: "{{ data.id | default(null) }}"
      appTitle: "{{ data.appTitle  | default('Home Assistant') }}"
      appIcon: "{{ data.appIcon | default('mdi:home-assistant') }}"
      color: "{{ data.color | default('#049cdb') }}"
      image: "{{ data.image | default(null) }}"
      smallIcon: "{{ data.smallIcon | default(null) }}"
      largeIcon: "{{ data.largeIcon | default(null) }}"
      corner: "{{ data.corner  | default(null) }}"
      seconds: "{{ data.seconds | default(null) }}"
      
  - name: TvOverlayNotifyFixed
    platform: rest
    method: POST_JSON
    resource: http://[ip]:[port]/notify_fixed
    verify_ssl: false
    title_param_name: title
    data:
      id: "{{ data.id | default(null) }}"
      text: "{{ data.text  | default(null) }}"
      icon: "{{ data.icon | default(null) }}"
      textColor: "{{ data.textColor | default(null) }}"
      iconColor: "{{ data.iconColor | default(null) }}"
      borderColor: "{{ data.borderColor | default(null) }}"
      backgroundColor: "{{ data.backgroundColor | default(null) }}"
      shape: "{{ data.shape | default(null) }}"
      visible: "{{ data.visible | default(true) }}"
      expiration: "{{ data.expiration | default(null) }}"

      
rest_command:
  tvoverlay:
    url: http://[ip]:[port]/set/overlay
    method: POST
    verify_ssl: false
    headers:
      accept: 'application/json'
    content_type:  'application/json; charset=utf-8'
    payload: '{{ payload }}'

Następnie można przetestować powiadomienie wywołując usługę w HA z menu Narzędzia deweloperskie -> Akcje

action: notify.tvoverlaynotify
data:
  message: The garage door has been open for 10 minutes.
  title: Your Garage Door Friend
  data:
    id: notification_sample
    appTitle: AppTitle
    appIcon: mdi:unicorn
    color: "#FFF000"
    smallIcon: mdi:speaker-multiple
    largeIcon: mdi:camera
    corner: bottom_end
    seconds: 20
    image: https://picsum.photos/300

Wysłanie powiadomienia z Frigate na telefon i TV

Jeżeli wysyłanie następuje tylko na telefon to można skorzystać z gotowego blueprinta (link).

Poniżej moja automatyzacja

- id: '1663062012156'
  alias: Frigate Notifications
  trigger:
    - platform: mqtt
      topic: frigate/events
      id: frigate-event
      payload: NAZWA_KAMERY
      value_template: "{{ value_json['after']['camera'] }}"
      variables:
        after_zones: "{{ trigger.payload_json['after']['entered_zones'] }}"
        before_zones: "{{ trigger.payload_json['before']['entered_zones'] }}"
        camera: "{{ trigger.payload_json['after']['camera'] }}"
        id: "{{ trigger.payload_json['after']['id'] }}"
        label: "{{ trigger.payload_json['after']['label'] }}"
        score: "{{ trigger.payload_json['after']['score'] }}"
        time_clip_start: "{{ trigger.payload_json['after']['start_time'] - 10.0 }}"
  # Conditions must be used to filter out objects / detections that are not of interest
  condition:
    # This condition is used to ensure that I am only notified of objects that are new or only just entered a zone. This is to avoid getting loads of updated notifications for the same dog in my backyard (as an example).
    - condition: or
      conditions:
        - condition: template
          value_template: "{{ trigger.payload_json['type'] == 'new' }}"
        - condition: template
          value_template: "{{ before_zones | length == 0 }}"
    - condition: template
      value_template: >-
            {{ 'Chodnik' in (after_zones + before_zones) }}
#    - condition: template
#      value_template: >-
#        {{
#         (as_timestamp(now())-as_timestamp(state_attr('automation.frigate_notifications_with_details2',
#            'last_triggered') | default(0)) | int > 5) }}
    - condition: state
      entity_id: input_boolean.tv_notify_motion
      state: 'on'

  action:
    - service: notify.mobile_phone_app
      data:
        message: >-
           {{ label }}({{ int(score | round(2) * 100) }}%) na {{ after_zones }} wykryty przez {{ camera }}, {{ id }}
        data:
          image: >-
                /api/frigate/notifications/{{ id }}/thumbnail.jpg
          url: >-
            https://HAOS_EXTERNAL_LINK/api/frigate/notifications/{{ id }}/{{camera}}/clip.mp4
          actions:
            - action: URI
              title: Video
              uri: >-
                https://HAOS_EXTERNAL_LINK/api/frigate/notifications/{{id}}/{{camera}}/clip.mp4
            - action: URI
              title: Snapshot
              uri: >-
                https://HAOS_EXTERNAL_LINK/api/frigate/notifications/{{ id }}/snapshot.jpg
            - action: URI
              title: Thumbnail
              uri: >-
                https://HAOS_EXTERNAL_LINK/api/frigate/notifications/{{ id }}/thumbnail.jpg
    - service: notify.tvoverlaynotify
      data:
        data:
          appTitle: Frigate
          largeIcon: mdi:camera
          color: "#000800"
          seconds: 30
          image: >-
             https://HAOS_EXTERNAL_LINK/api/frigate/notifications/{{id}}/thumbnail.jpg
        title: >-
          {{ label }} wykryto na {{ camera }}
        message: >-
          {{ label }}({{ int(score | round(2) * 100) }}%) na {{ after_zones }} wykryty przez {{ camera }}
  mode: parallel
  max: 10
#  trace:
#    stored_traces: 20

Linki

https://frigate.video

https://docs.frigate.video

https://github.com/gugutab/TvOverlay

https://github.com/SgtBatten/HA_blueprints

https://community.home-assistant.io/t/frigate-mobile-app-notifications-2-0/559732/2618

Komentarze

Dodaj komentarz

Twój adres e-mail nie zostanie opublikowany. Wymagane pola są oznaczone *

Więcej wpisów