HyperAST          

Book GUI Rust doc Getting Started Compute code Metrics(GUI) Track Code(GUI)

Summary

The HyperAST project aims to facilitate the exploration and processing of large source code histories. At its core an HyperAST is a compressed syntax tree structure similar to the MerkleDAG of Git (but beyond files). An HyperAST is typically constructed from a Git repository, by parsing each unique file once with TreeSitter. In the process, only structurally unique subtrees are stored and processed, thus compressing and incrementally deriving useful information to be used later.

The project includes a use-def solver, based on the context-free indexing of references present in subtrees (each subtree has a bloom filter of contained references). It should be noted that this feature is currently being reworked to generalize beyond the analysis of Java projects.

It also includes a tree diff module, called HyperDiff, largely inspired by Gumtree and its algorithms. We notably adapted the algorithms to lazily decompress nodes.

Many other features are provided, and can be discovered reading the book, the rust doc or through the graphical app.

How to use

You can use the dedicated GUI in your browser. However, in order to use any of the GUI features, you will need to launch/connect to the server.

Launch the server with Cargo

# from the project root dir, after having cloned the repository
cargo run -p backend --release 

Warning

You have to handle system dependencies yourself, such as, rustc, openssl. Please use rustup to automatically handle the build chain compatibility.

Launch server with Nix

nix run .#hyperast-backend

instead of cargo run, or without even cloning the repository

nix run github:HyperAST/HyperAST#hyperast-webapi # Nix handles everything!

Nix is a package manager for reproducible, declarative and reliable systems. This will download all dependencies and build locally. It works easily on most *NIX systems (Linux, WSL, macOs, ...) and can produce OCI-images like Docker.

Tip

Look there for instruction on how to install Nix on your system.

There is also a development shell provided with all the necessary dependencies installed in a healthy environment to develop and build the project. You can enter the environment with:

nix develop # from the project root dir

How to Cite

If you use HyperAST and HyperDiff for academic purposes, please cite the following papers:

@inproceedings{ledilavrec:hal-03764541,
  TITLE = {{HyperAST: Enabling Efficient Analysis of Software Histories at Scale}},
  AUTHOR = {Le Dilavrec, Quentin and Khelladi, Djamel Eddine and Blouin, Arnaud and J{\'e}z{\'e}quel, Jean-Marc},
  URL = {https://hal.inria.fr/hal-03764541},
  BOOKTITLE = {{ASE 2022 - 37th IEEE/ACM International Conference on Automated Software Engineering}},
  PUBLISHER = {{IEEE}},
  PAGES = {1-12},
  YEAR = {2022}
}

@inproceedings{ledilavrec:hal-04189855,
  TITLE = {{HyperDiff: Computing Source Code Diffs at Scale}},
  AUTHOR = {Le Dilavrec, Quentin and Khelladi, Djamel Eddine and Blouin, Arnaud and J{\'e}z{\'e}quel, Jean-Marc},
  URL = {https://inria.hal.science/hal-04189855},
  BOOKTITLE = {{ESEC/FSE 2023 - 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering}},
  PUBLISHER = {{ACM}},
  PAGES = {1-12},
  YEAR = {2023}
}