Skip to content
juancespedes.dev
ES: Leer esta página en español
All projects
Case studyAnonymized

Real-time power generation telemetry

I built the service that reads five energy meters every 2 seconds, integrates energy per hourly period, compares it against the market dispatch and feeds a live dashboard and a Power BI report.

Project facts

Role
Data and backend engineer, sole developer
Period
Mar 2026 to present
Public links
Internal system: the code is private, and names, data and diagrams were left out or recreated.

  • 2 s

    read interval across 5 meters over Modbus TCP, with 15 to 25 ms p50 latencySource: Service configuration and latency measurements, 2026

  • 300 MB to 5 MB

    smaller deployment after moving from an automated browser to reading the meters directlySource: Project architecture decision log

  • 410+

    automated test cases across Vitest and pytestSource: Repository test suites, September 2026

The problem

Operations needed to see, on a single screen, the actual output of four generating units against the market operator's dispatch, and the existing readings depended on a centralized system that was a single point of failure.

My role

  • Designed direct meter reading, per-period energy integration and the comparison against market dispatch.
  • Built the Node.js backend, the React dashboard and the Python service that feeds the data lakehouse.
  • Statistically validated the backup meters and documented the sign convention of each metering point.
  • Run two production instances, one per plant, from the same build.

Constraints

  • Readings every 2 seconds without losing integrated energy if the service restarts.
  • Two plants with different sign conventions depending on the metering point.
  • A failed meter must not pollute the totals with false zeros.

Decisions

  1. 01

    Read the meters directly over Modbus TCP

    Removes the single point of failure and the automated browser from the deployment, and makes the latency of every reading measurable.

    Ruled out

    • Keep scraping the centralized system's web interface
  2. 02

    Null means no reading, never zero

    Telling missing data apart from a zero value makes a failed meter detectable instead of reporting zero generation. The rule lives in a domain module shared by the backend and the dashboard.

    Ruled out

    • Fill missing readings with zero
  3. 03

    Statistical validation before using the backup meters

    A one-hour parallel capture with bias, p95 and integrated energy criteria proved the backup meters are interchangeable.

    Ruled out

    • Assume the backup meters read the same
  4. 04

    One build for two plants, configured at load time

    Each plant's branding and unit order are read from a configuration file at runtime, so a fix ships once and serves both.

    Ruled out

    • Compile a separate dashboard per plant

Architecture

A Node.js service reads the meters, integrates energy and compares it with the market operator's files and the redispatch emails; the dashboard receives everything over WebSockets. In parallel, a Python service writes the same telemetry to the lakehouse behind Power BI.

  1. Sources

    • Energy meters5 meters, Modbus TCP every 2 s
    • Market operator filesDaily dispatch and redispatch every 5 min
    • Redispatch emailsRead with Microsoft Graph

  2. Processing

    • Meter readerFailures isolated per unit
    • Energy integratorMW to MWh per hourly period
    • Python telemetry serviceDelta tables without Spark

  3. Storage

    • SQL ServerPeriods, dispatch and change history
    • Microsoft Fabric lakehouseMirrored to PostgreSQL

  4. Consumers

    • Live dashboardReact over WebSockets
    • Power BI report

How data moves

  1. Energy meterstoMeter readerReadings every 2 s
  2. Meter readertoEnergy integratorPower per unit, null if a meter is missing
  3. Market operator filestoEnergy integratorScheduled dispatch per period
  4. Redispatch emailstoEnergy integratorFinal dispatch for the period
  5. Energy integratortoSQL ServerCheckpoint every 30 s
  6. Energy integratortoLive dashboardOutput, projection and deviation
  7. Energy meterstoPython telemetry serviceIndependent reading
  8. Python telemetry servicetoMicrosoft Fabric lakehouseWrite every 15 s
  9. Microsoft Fabric lakehousetoPower BI reportCorporate report model

Results

  • Operations sees on one screen the actual output of four units against dispatch, with a projection for the current period.
  • The single point of failure is gone and read latency dropped from about 1.1 s to 15 to 25 ms (p50).
  • After a restart, the service rebuilds closed periods from the database and loses no integrated energy.
  • The same telemetry feeds a corporate Power BI report without touching the dashboard.

What I would do differently

  • I would measure latency and the rate of null readings from day one; those figures justified the protocol change and would have brought the decision forward.
  • I would keep a single meter-reading service that publishes to both consumers, instead of maintaining the same topology in Node.js and in Python.

Technologies

  • Node.js
  • Python
  • Modbus TCP
  • WebSockets
  • SQL Server
  • Microsoft Fabric
  • PostgreSQL
  • Power BI
  • Microsoft Graph
  • React
  • Linux
  • systemd