Home Assistant Observability with Fluent Bit and Seq

Managing a smart home with Home Assistant can quickly become complex. With numerous devices, automations, and integrations generating logs, having a centralized way to search, filter, and analyze these logs is crucial. In this post, I’ll share how I built an observability stack using Fluent Bit for log forwarding and Seq for centralized log management.

Why Logging Matters in a Smart Home

Logs provide invaluable insights into how your smart home system is functioning. They help in diagnosing issues, understanding automation behavior, and monitoring system performance. However, Home Assistant doesn’t offer a straightforward way to send logs to a centralized server—likely because most users wouldn’t require such capabilities.

That’s why I decided to centralize my logs using Seq. It’s lightweight, easy to use, and free for personal use—making it an excellent choice for a home setup.

My Observability Stack

Fluent Bit Add-on for Home Assistant

I discovered a lesser-known add-on that deploys Fluent Bit, a powerful log processing engine. Fluent Bit acts as a log forwarder, collecting logs from Home Assistant and sending them to Seq. Here’s how I set it up:

GitHub - ablyler/ha-addon-fluent-bit: Home Assistant Addon for Fluent Bit

Home Assistant Addon for Fluent Bit. Contribute to ablyler/ha-addon-fluent-bit development by creating an account on GitHub.

GitHubablyler

• Install the add-on: You'll need to add the repository to the Add-on store and then for it and follow the installation instructions. Make sure to start the add-on once so that the configuration directory is created and then follow the instructions. My addon config looks like this (and the actual file is in addon_configs/144431fc_fluent_bit/fluent_bit.conf):

config_file: fluent_bit.conf

• Configure log inputs: My setup captures logs directly from the systemd journal to ensure comprehensive log collection:

[INPUT]
    Name systemd
    Path /var/log/journal
    DB /data/fluent-bit.db

This method provides a reliable way to capture logs from both Home Assistant and the underlying OS.

Processing Logs Efficiently

Capturing logs is just the first step. Processing them efficiently to remove noise and extract meaningful information is where Fluent Bit shines. My configuration includes:

• Filtering out unnecessary logs: For example, I exclude logs from DNS and Fluent Bit itself to reduce noise:

[FILTER]
    Name    grep
    Match   *
    Logical_Op or
    Exclude CONTAINER_NAME hassio_dns
    Exclude CONTAINER_NAME addon_144431fc_fluent_bit

• Extracting log levels with Lua: To categorize logs by severity, I use a Lua script to extract log levels like ERROR, WARNING, and INFO. I also use the same script to clean the log messages from ANSI color coding that will show up as gibberish when it lands in Seq.

[FILTER]
    Name    lua
    Match   *
    Script  extract_level.lua
    Call    extract_level

function clean_message(str)
    -- Pattern to match ANSI escape sequences
    local ansi_pattern = '\27%[[%d;]+m'
    -- Remove ANSI codes
    str = string.gsub(str, ansi_pattern, '')
    -- Remove \u0001 and \u0002 characters
    str = string.gsub(str, '\001', '')
    str = string.gsub(str, '\002', '')
    return str
end

function extract_level(tag, timestamp, record)
    if record["@m"] then
        -- Clean the message
        record["@m"] = clean_message(record["@m"])
        
        -- Extract log level
        local level = string.match(record["@m"], "%s(%u+)%s+%(")
        if level then
            record["@l"] = level
        end
    end
    return 2, timestamp, record
end

Centralized Log Management with Seq

With logs cleaned and formatted, I forward them to my Seq instance for centralized management. Here’s why I chose Seq:

• Powerful search and filtering: Makes it easy to find logs based on keywords or log levels.
• Custom dashboards: I created dashboards to monitor key metrics like failed automations and system errors.
• Alerts: Configured alerts for critical log patterns, such as repeated integration failures.

[OUTPUT]
    Name             http
    Match            *
    Host             seq.internal.shymoose.com
    Port             5341
    URI              /api/events/raw?clef
    Format           json_lines
    Json_date_key    @t
    Json_date_format iso8601

Putting it all together, the fluent_bit.conf file looks like this:

[SERVICE]
    Parsers_File  parsers.conf

[INPUT]
    Name systemd
    Path /var/log/journal
    DB /data/fluent-bit.db

[FILTER]
    Name    grep
    Match   *
    Regex   CONTAINER_NAME .+

[FILTER]
    Name    grep
    Match   *
    Logical_Op or
    Exclude CONTAINER_NAME hassio_dns
    Exclude CONTAINER_NAME addon_144431fc_fluent_bit

[FILTER]
    Name    modify
    Match   *
    Rename  MESSAGE @m

[FILTER]
    Name    lua
    Match   *
    Script  extract_level.lua
    Call    extract_level

[OUTPUT]
    Name             http
    Match            *
    Host             seq.internal.shymoose.com
    Port             5341
    URI              /api/events/raw?clef
    Format           json_lines
    Json_date_key    @t
    Json_date_format iso8601
    Log_response_payload False

Conclusion

Centralizing Home Assistant logs using Fluent Bit and Seq has significantly improved how I manage and monitor my smart home. The ability to filter out noise, extract useful information with Lua scripts, and visualize logs in real-time has been a game changer. If you’re looking to gain deeper insights into your Home Assistant setup, this approach is worth exploring.

Happy logging!