Skip to content

mzabani/hpgsql

Repository files navigation

CI

Hpgsql is a PostgreSQL driver written in pure Haskell (no libpq), with an API largely inspired by the great postgresql-simple library, but featuring:

  • Usage of PostgreSQL's binary protocol
  • Query arguments passed via the protocol instead of being escaped into the query string
  • Pipelining
  • Prepared statements
  • Ability to stream query results directly from the socket (not just with cursors)
  • Interruption safety, except for very specific (and documented) edge cases
  • Thread safety, unless specific (and documented) instructions say otherwise
  • A SQL quasiquoter like the one in postgresql-query and hasql-interpolate

Here's an example of a pipeline mixing streams, prepared and non prepared statements:

f :: Int -> IO (Stream (Of Aeson.Value) IO ())
f val = do
  (updateTbl :: IO (), aggRes :: IO (Only Int), largeResults) <-
    runPipeline conn $
      (,,)
        <$> pipelineExec_ [sql|UPDATE tbl SET val=#{val}|]
        <*> pipeline1 [sql|SELECT SUM(val) FROM tbl|]
        <*> pipelineSWith
          (rowDecoder @(Vector Int, Vector Text))
          -- We use a prepared statement for the query below
          [sqlPrep|SELECT x, y FROM tbl|]
  updateTbl
  Only total <- aggRes
  Streaming.map Aeson.toJSON <$> largeResults

Current status

Hpgsql is in active development, is new and has not been used in Production yet. It currently lacks (at least) many of the authentication methods that libpq provides; for now only cleartext and MD5 password auth are supported, and only unencrypted connections.

That being said, one of my other projects, codd, has a tests-passing branch using hpgsql-simple-compat (the nearly API compatible fork of postgresql-simple described in the next section) with very few modifications.

Migrating from postgresql-simple

This repository contains a fork of postgresql-simple that preserves as much as possible** the names of modules, functions, types, classes, exceptions, etc. Its purpose is to ease migrating to hpgsql, and I intend to fully support its development to make it and keep it as similar to postgresql-simple as it can be.

It is called hpgsql-simple-compat, and its implementation uses hpgsql. You can get a HPgConnection out of it so you can gradually migrate your queries to hpgsql.

It also contains parts of postgresql-libpq and postgresql-query, all implemented on top of hpgsql.

You should start by swapping all of "postgresql-simple", "postgresql-libpq", and "postgresql-query" in your cabal files by "hpgsql-simple-compat". But read MIGRATING.md for tips and gotchas.

  • I haven't been able to preserve everything, so some differences do exist. Also the library is not feature complete yet.
  • There may be small intentional differences added to help the transition, like a sqlStatement field in the SqlError exception so it's easier to know which queries are failing.

Performance

Some benchmarks show materializing large query results with hpgsql takes ~38% the time postgresql-simple takes, and ~70% the time hasql takes (on my computer, Linux x64, GHC 9.10.3, compiled with -O1).

When comparing hpgsql's Stream querying, hpgsql takes ~13% the time of both streaming-postgresql-simple and postgresql-simple's cursor folding functions, although this might not be a fair comparison for some use cases.

hpgsql's binary COPY runs in about the same time as postgresql-simple's textual COPY.

Peak allocated memory is harder to analyze.

See BENCHMARKS.md for more details.

Contributing

Working with this repository

Once you clone this repository, you will need Nix and optionally direnv. Then you can run direnv allow or nix-shell in the project's root and you will have everything you need to build, run tests and benchmarks, locally.

I recommend you run run list to see what's available, but I'll paste a sample here as of 2026-04-22 as well:

$ run list
Commands:
  list            (builtin) List available commands
  help            (builtin) Show help for a command
  version         (builtin) Show run version
  benchmarks      Runs benchmarks with a postgresql DB listening.
  bench-single    Runs the benchmarks executable without all the CSV-producing and memory usage collecting tooling around it.
  ci-tests        Runs all tests that CI runs, exactly like CI runs them.
  format-hs       Formats all Haskell files with fourmolu.
  tests           Runs all tests.
  tests-stress    Runs tests 100 times, reporting how many passed and how many failed.
  tests-compat    Runs hpgsql-simple-compat's tests.

You don't need to install and configure PostgreSQL yourself: all the commands start temporary instances of PostgreSQL as necessary, and stop them at the end, including e.g. run tests.

But you can run pg_ctl start and then psql postgres to play with a local instance. Use pg_ctl stop to stop it.

For example:

$ run tests -- --match Pipelining # No need to start postgres as it happens automatically

CI pipeline

Hpgsql's CI pipeline runs tests against all major supported versions of PostgreSQL, on Linux and Mac, and also tests the multi-threaded and single-threaded RTS.

It also runs hpgsql-simple-compat's (the API-compatible fork of postgresql-simple that uses Hpgsql internally) test suite, which is a slightly modified subset of postgresql-simple's own test suite.

My hope is that you can have a reasonable level of assurance that your contributions are sound.

About

A pure Haskell (no libpq) postgresql driver that draws inspiration from postgresql-simple, has streaming built into its core, speaks postgresql's binary protocol, has pipelining, interruption safety, and (reasonable) thread safety.

Topics

Resources

Stars

Watchers

Forks

Contributors