diff --git a/.gitignore b/.gitignore index fc8ef955b..5689b87f9 100644 --- a/.gitignore +++ b/.gitignore @@ -6,4 +6,11 @@ build/ *.swp doxygen-docs/ +docs/_build/ +docs/api/generated/ +docs/api/reference/ +docs/examples/generated/ +docs/generated/ +docs/models/generated/ +docs/xml/ __pycache__ diff --git a/.readthedocs.yaml b/.readthedocs.yaml new file mode 100644 index 000000000..506264ffd --- /dev/null +++ b/.readthedocs.yaml @@ -0,0 +1,20 @@ +version: 2 + +build: + os: ubuntu-24.04 + tools: + python: "3.12" + apt_packages: + - doxygen + - graphviz + jobs: + pre_build: + - python docs/generate_model_docs.py + - cd docs && doxygen Doxyfile + +sphinx: + configuration: docs/conf.py + +python: + install: + - requirements: docs/requirements.txt diff --git a/CHANGELOG.md b/CHANGELOG.md index 066786e09..612211975 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ - Added 3, 10, 37, and 39 bus test cases. - Updated documentation. +- Added support for Read the Docs - Added JSON parsing. - Automatic differentiation with enzyme (w.r.t. internal and external variables). - Added PR and issue templates. diff --git a/docs/Doxyfile b/docs/Doxyfile new file mode 100644 index 000000000..8564e075e --- /dev/null +++ b/docs/Doxyfile @@ -0,0 +1,17 @@ +PROJECT_NAME = "GridKit" +OUTPUT_DIRECTORY = . + +INPUT = ../GridKit +RECURSIVE = YES +EXCLUDE_PATTERNS = *.md +FULL_PATH_NAMES = YES +STRIP_FROM_PATH = .. + +GENERATE_HTML = NO +GENERATE_LATEX = NO +GENERATE_XML = YES +XML_OUTPUT = xml +XML_PROGRAMLISTING = NO + +EXTRACT_ALL = YES +QUIET = YES diff --git a/docs/README.md b/docs/README.md index 2242f220a..923ca5d1a 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,32 +1,39 @@ -# GridKit Documentation Target +# GridKit Documentation -We use CMake's built-in `FindDoxygen` module to generate a target that will -use Doxygen to build the project documentation. +GridKit documentation can be built two ways: -## Building +1. Read the Docs builds the Sphinx site from `docs/conf.py`. +2. CMake can build the existing Doxygen HTML target. + +## Read the Docs Build + +The Read the Docs proof of concept is configured by `.readthedocs.yaml`. +Before Sphinx runs, the build generates Markdown wrapper pages and Doxygen XML: -The documentation target is excluded from the default set of targets built via -`make` (or `cmake --build .`). It must be designated explicitly with ```sh -make GridKitDocs +python docs/generate_model_docs.py +cd docs && doxygen Doxyfile ``` -or + +To test the same flow locally: + ```sh -cmake --build . -t GridKitDocs +python -m pip install -r docs/requirements.txt +python docs/generate_model_docs.py +cd docs && doxygen Doxyfile +cd .. +sphinx-build -T -b html docs docs/_build/html ``` -The generated files can be found in the build directory under `docs/html/` and -the main page is `docs/html/index.html`. - -If you wish to inspect the Doxyfile itself, it is generated as -`docs/Doxyfile.GridKitDocs`. - -## Notes -The reasoning behind taking this approach is as follows: - -1. It is easier to maintain just the few options we need to customize rather - than a whole Doxyfile (leave what can be generated out of the repo); if - another option is needed, it's easy to look it up. -2. Since the documentation can be seen as a "product" or "artifact" of the code, - it makes sense for it to be a buildable "target" -3. Generated files should not be placed in the source directory. The binary - directory makes sense for this, and the CMake target makes this easy. + +The generated Sphinx files, Doxygen XML, and HTML output are build artifacts and +should not be committed. + +## CMake Doxygen Target + +The existing CMake target is still available for standalone Doxygen HTML output: + +```sh +cmake --build build -t GridKitDocs +``` + +The generated files are written under the build directory. diff --git a/docs/api.md b/docs/api.md new file mode 100644 index 000000000..39b30249b --- /dev/null +++ b/docs/api.md @@ -0,0 +1,35 @@ +# API Reference + +```{toctree} +:maxdepth: 1 +:titlesonly: +:hidden: + +GridKit Core +Solvers +Phasor Dynamics +Power Electronics +Power Flow Data +Linear Algebra +Math Utilities +Model Utilities +Utilities +Testing +``` + +The API reference is organized by the primary GridKit namespaces. Lower-level +class, struct, enum, function, and file pages are generated by Exhale and linked +from these namespace pages. + +## Main Areas + +- [GridKit core](api/reference/namespace_GridKit.rst) +- [Solvers](api/reference/namespace_AnalysisManager.rst) +- [GridKit::PhasorDynamics](api/reference/namespace_GridKit__PhasorDynamics.rst) +- [GridKit::PowerElectronics](api/reference/namespace_GridKit__PowerElectronics.rst) +- [GridKit::PowerFlowData](api/reference/namespace_GridKit__PowerFlowData.rst) +- [GridKit::LinearAlgebra](api/reference/namespace_GridKit__LinearAlgebra.rst) +- [GridKit::Model](api/reference/namespace_GridKit__Model.rst) +- [GridKit::Math](api/reference/namespace_GridKit__Math.rst) +- [GridKit::Utilities](api/reference/namespace_GridKit__Utilities.rst) +- [GridKit::Testing](api/reference/namespace_GridKit__Testing.rst) diff --git a/docs/applications/index.md b/docs/applications/index.md new file mode 100644 index 000000000..14211ecb9 --- /dev/null +++ b/docs/applications/index.md @@ -0,0 +1,15 @@ +# Applications + +```{toctree} +:maxdepth: 2 +:titlesonly: +:hidden: + +../generated/application-input-format +``` + +## PhasorDynamics + +- [Application Input Format](../generated/application-input-format.md) +- `application/PhasorDynamics/DynamicSimulation.cpp` +- `application/PhasorDynamics/ContingencyAnalysis.cpp` diff --git a/docs/conf.py b/docs/conf.py new file mode 100644 index 000000000..3bf0b78d5 --- /dev/null +++ b/docs/conf.py @@ -0,0 +1,63 @@ +from pathlib import Path + +from exhale import utils as exhale_utils + +project = "GridKit" +author = "GridKit Developers" + +docs_dir = Path(__file__).parent.resolve() + +extensions = ["breathe", "exhale", "myst_parser", "sphinx.ext.graphviz"] + +breathe_projects = {"GridKit": str(docs_dir / "xml")} +breathe_default_project = "GridKit" + + +def exhale_specs(kind): + if kind in {"class", "struct"}: + return [":members:"] + return [] + + +exhale_args = { + "containmentFolder": "./api/reference", + "rootFileName": "EXCLUDE", + "doxygenStripFromPath": str(docs_dir.parent), + "createTreeView": False, + "customSpecificationsMapping": exhale_utils.makeCustomSpecificationsMapping( + exhale_specs + ), + "contentsDirectives": False, + "fullToctreeMaxDepth": 1, + "pageLevelConfigMeta": ":orphan:", +} + +primary_domain = "cpp" + +html_theme = "sphinx_rtd_theme" +html_extra_path = ["Figures"] +html_theme_options = { + "collapse_navigation": False, + "navigation_depth": 6, + "titles_only": True, +} + +source_suffix = { + ".rst": "restructuredtext", + ".md": "markdown", +} + +myst_enable_extensions = [ + "amsmath", + "dollarmath", + "html_image", +] +myst_heading_anchors = 5 + +exclude_patterns = [ + "_build", + "api/generated/**", + "xml", + "README.md", + "superpowers/**", +] diff --git a/docs/development/index.md b/docs/development/index.md new file mode 100644 index 000000000..12d13f933 --- /dev/null +++ b/docs/development/index.md @@ -0,0 +1,19 @@ +# Development + +```{toctree} +:maxdepth: 2 +:titlesonly: +:hidden: + +../generated/contributing +../generated/buildsystem +../generated/documentation-build +../generated/changelog +``` + +## Sections + +- [Contributing](../generated/contributing.md) +- [Buildsystem](../generated/buildsystem.md) +- [Documentation Build](../generated/documentation-build.md) +- [Changelog](../generated/changelog.md) diff --git a/docs/generate_model_docs.py b/docs/generate_model_docs.py new file mode 100644 index 000000000..555155b92 --- /dev/null +++ b/docs/generate_model_docs.py @@ -0,0 +1,386 @@ +#!/usr/bin/env python3 +"""Generate Sphinx pages from selected GridKit Markdown files.""" + +from __future__ import annotations + +import os +import re +import shutil +import subprocess +from pathlib import Path + + +ROOT = Path(__file__).resolve().parents[1] +DOCS = ROOT / "docs" +MODEL = ROOT / "GridKit" / "Model" +EXAMPLES = ROOT / "examples" +GITHUB_REPO = "https://github.com/ORNL/GridKit" + +GENERATED = DOCS / "generated" +GENERATED_MODELS = DOCS / "models" / "generated" +GENERATED_EXAMPLES = DOCS / "examples" / "generated" + +COMMON_MATH = ROOT / "GridKit" / "CommonMath.md" +PHASOR_INPUT_FORMAT = MODEL / "PhasorDynamics" / "INPUT_FORMAT.md" + +PROJECT_PAGES = { + ROOT / "README.md": (GENERATED / "readme.md", "GridKit"), + ROOT / "INSTALL.md": (GENERATED / "install.md", "Installation"), + ROOT / "CONTRIBUTING.md": (GENERATED / "contributing.md", "Contributing"), + ROOT / "CHANGELOG.md": (GENERATED / "changelog.md", "Changelog"), + DOCS / "README.md": (GENERATED / "documentation-build.md", "Documentation Build"), + ROOT / "buildsystem" / "README.md": (GENERATED / "buildsystem.md", "Buildsystem"), + ROOT / "application" / "PhasorDynamics" / "README.md": ( + GENERATED / "application-input-format.md", + "Application Input Format", + ), +} + +MODEL_READMES = tuple(path.resolve() for path in sorted(MODEL.glob("**/README.md"))) +EXAMPLE_READMES = tuple(path.resolve() for path in sorted(EXAMPLES.glob("**/README.md"))) + + +def git_output(*args: str) -> str: + try: + return subprocess.check_output( + ["git", *args], + cwd=ROOT, + stderr=subprocess.DEVNULL, + text=True, + ).strip() + except (OSError, subprocess.CalledProcessError): + return "" + + +def source_ref() -> str: + if os.environ.get("READTHEDOCS_VERSION_TYPE") == "external": + if ref := os.environ.get("READTHEDOCS_GIT_COMMIT_HASH"): + return ref + if ref := os.environ.get("READTHEDOCS_GIT_IDENTIFIER"): + return ref + if ref := git_output("rev-parse", "--abbrev-ref", "HEAD"): + if ref != "HEAD": + return ref + return git_output("rev-parse", "HEAD") or "HEAD" + + +SOURCE_REF = source_ref() + + +def slug(text: str) -> str: + text = re.sub(r"([a-z0-9])([A-Z])", r"\1-\2", text) + text = re.sub(r"[^A-Za-z0-9.-]+", "-", text.replace("_", "-").replace("/", "-")) + return re.sub(r"-+", "-", text).strip("-").lower() + + +def relative(target: Path, page: Path) -> str: + return Path(os.path.relpath(target, page.parent)).as_posix() + + +def model_page(readme: Path) -> Path: + parts = [slug(part) for part in readme.parent.relative_to(MODEL).parts] + return GENERATED_MODELS.joinpath(*parts, "index.md") + + +def example_page(directory: Path) -> Path: + parts = [slug(part) for part in directory.relative_to(EXAMPLES).parts] + return GENERATED_EXAMPLES.joinpath(*parts, "index.md") + + +SOURCE_TO_PAGE = {source.resolve(): output for source, (output, _) in PROJECT_PAGES.items()} +SOURCE_TO_PAGE[COMMON_MATH.resolve()] = GENERATED_MODELS / "common-math.md" +SOURCE_TO_PAGE[PHASOR_INPUT_FORMAT.resolve()] = ( + GENERATED_MODELS / "phasor-dynamics" / "input-format.md" +) +SOURCE_TO_PAGE.update({readme: model_page(readme) for readme in MODEL_READMES}) +SOURCE_TO_PAGE.update({readme: example_page(readme.parent) for readme in EXAMPLE_READMES}) + + +def example_dirs() -> set[Path]: + dirs = {EXAMPLES.resolve()} + for readme in EXAMPLE_READMES: + directory = readme.parent + while directory != EXAMPLES.parent: + dirs.add(directory.resolve()) + if directory == EXAMPLES: + break + directory = directory.parent + return dirs + + +EXAMPLE_DOC_DIRS = example_dirs() + + +def title_for(source: Path) -> str: + if source in PROJECT_PAGES: + return PROJECT_PAGES[source][1] + if source == COMMON_MATH: + return "CommonMath" + if source == PHASOR_INPUT_FORMAT: + return "Input Format" + if source in EXAMPLE_READMES and source.parent == EXAMPLES: + return "Examples" + return source.parent.name + + +def toctree(entries: list[str], maxdepth: int = 2) -> str: + return ( + "```{toctree}\n" + f":maxdepth: {maxdepth}\n" + ":titlesonly:\n:hidden:\n\n" + + "\n".join(entries) + + "\n```\n" + ) + + +def children_of(directory: Path, readmes: tuple[Path, ...]) -> list[Path]: + return sorted(readme.parent for readme in readmes if readme.parent.parent == directory) + + +def example_children(directory: Path) -> list[Path]: + return sorted(path for path in EXAMPLE_DOC_DIRS if path.parent == directory and path != directory) + + +def generated_toctree(source: Path, page: Path) -> str: + entries = [] + if source in MODEL_READMES: + if source == MODEL / "PhasorDynamics" / "README.md": + entries.append(relative(SOURCE_TO_PAGE[PHASOR_INPUT_FORMAT.resolve()].with_suffix(""), page)) + entries += [ + relative(model_page(child / "README.md").with_suffix(""), page) + for child in children_of(source.parent, MODEL_READMES) + ] + elif source in EXAMPLE_READMES: + entries = [ + relative(example_page(child).with_suffix(""), page) + for child in example_children(source.parent) + ] + return f"{toctree(entries)}\n" if entries else "" + + +def split_anchor(target: str) -> tuple[str, str]: + path, sep, anchor = target.partition("#") + return path, f"{sep}{anchor}" if sep else "" + + +def is_external(target: str) -> bool: + return bool(re.match(r"^[A-Za-z][A-Za-z0-9+.-]*:", target)) + + +def fallback(source_dir: Path, target: str) -> Path | None: + candidates = [] + if target.endswith("src/Model/PowerFlow/Gen/README.md"): + candidates.append(MODEL / "PowerFlow" / "README.md") + if target.startswith("src/Model/"): + candidates.append(ROOT / "GridKit" / target.removeprefix("src/")) + if target.startswith("Model/"): + candidates.append(ROOT / "GridKit" / target) + if "Model/" in target: + candidates.append(MODEL / target.split("Model/", 1)[1]) + if source_dir.relative_to(ROOT).parts[:1] == ("application",): + candidates.append(ROOT / "GridKit" / target.removeprefix("../../")) + return next((path.resolve() for path in candidates if path.exists()), None) + + +def page_link(path: Path, anchor: str, current_page: Path) -> str | None: + path = path.resolve() + if path == COMMON_MATH.resolve() and anchor == "#anti-windup-indicator": + anchor = "#derived-functions" + if path in SOURCE_TO_PAGE: + return f"{relative(SOURCE_TO_PAGE[path], current_page)}{anchor}" + readme = (path / "README.md").resolve() if path.is_dir() else None + if readme in SOURCE_TO_PAGE: + return f"{relative(SOURCE_TO_PAGE[readme], current_page)}{anchor}" + return None + + +def rewrite_link(source_dir: Path, target: str, current_page: Path) -> str: + if not target or target.startswith("#") or is_external(target): + return target + path_text, anchor = split_anchor(target) + path = (source_dir / path_text).resolve() + if not path.exists(): + path = fallback(source_dir, path_text) or path + link = page_link(path, anchor, current_page) + if link: + return link + if path.exists() and path.is_dir(): + return f"{GITHUB_REPO}/tree/{SOURCE_REF}/{path.relative_to(ROOT).as_posix()}{anchor}" + if path.exists() or ROOT in path.parents: + return f"{relative(path, current_page)}{anchor}" + return target + + +def rewrite_asset(source_dir: Path, target: str, current_page: Path) -> str: + if not target or target.startswith("#") or is_external(target): + return target + path_text, anchor = split_anchor(target) + return f"{relative((source_dir / path_text).resolve(), current_page)}{anchor}" + + +def rewrite_markdown_links(text: str, source_dir: Path, current_page: Path) -> str: + text = re.sub( + r"!\[([^\]]*)\]\(([^)\s]+)\)", + lambda m: f"![{m.group(1)}]({rewrite_asset(source_dir, m.group(2), current_page)})", + text, + ) + return re.sub( + r"(? str: + text = re.sub(r"(?im)^\s*]*\balign=[\"']center[\"'][^>]*>\s*$", "", text) + text = re.sub(r"(?im)^\s*\s*$", "", text) + + def attr(attrs: str, name: str) -> str | None: + match = re.search(rf"\b{name}\s*=\s*([\"'])(.*?)\1", attrs, re.IGNORECASE) + return match.group(2) if match else None + + def image(match: re.Match[str]) -> str: + before, target, after = match.groups() + attrs = f"{before} {after}" + options = [] + if alt := attr(attrs, "alt"): + options.append(f":alt: {alt}") + if (align := attr(attrs, "align")) in {"left", "center", "right"}: + options.append(f":align: {align}") + options_text = "\n".join(options) + if options_text: + options_text += "\n" + target = rewrite_asset(source_dir, target, current_page) + return f"\n```{{image}} {target}\n{options_text}```\n" + + return re.sub( + r"]*)\bsrc=[\"']([^\"']+)[\"']([^>]*)>", + image, + text, + flags=re.IGNORECASE, + ) + + +def normalize_headings(text: str) -> str: + levels = sorted({len(m.group(1)) for m in re.finditer(r"(?m)^(#{1,6})(\s+)", text)}) + levels = {level: index + 2 for index, level in enumerate(levels)} + return re.sub( + r"(?m)^(#{1,6})(\s+)", + lambda m: f"{'#' * levels.get(len(m.group(1)), len(m.group(1)))}{m.group(2)}", + text, + ) + + +def strip_title(text: str, title: str) -> str: + def comparable(value: str) -> str: + return re.sub(r"[^a-z0-9]+", "", value.replace("™", "").lower()) + + match = re.match(r"\s*#\s+(.+?)\s*(?:\n+|$)", text) + if match and comparable(match.group(1)) == comparable(title): + return text[match.end() :] + return text + + +def normalize_comment_fences(text: str, source: Path) -> str: + if source != ROOT / "CONTRIBUTING.md": + return text + + def fence(match: re.Match[str]) -> str: + lines = [line.strip() for line in match.group(1).splitlines() if line.strip()] + if lines and all(line.startswith(("/", "*")) for line in lines): + return f"```text\n{match.group(1)}```" + return match.group(0) + + return re.sub(r"```c\+\+\n(.*?)```", fence, text, flags=re.DOTALL) + + +def normalize_markdown(text: str, source: Path, title: str) -> str: + text = strip_title(text, title) + text = normalize_comment_fences(text, source) + text = re.sub(r"(?m)^```\s*math\s*$", "```{math}", text) + text = re.sub(r"\$`([^`]+)`\$", r"$\1$", text) + text = normalize_headings(text) + if source == COMMON_MATH: + text = re.sub( + r"(?m)^(#{2,6}\s+Derived Functions)", + r"(anti-windup-indicator)=\n\1", + text, + count=1, + ) + return text + + +def write_page(source: Path) -> None: + source = source.resolve() + page = SOURCE_TO_PAGE[source] + page.parent.mkdir(parents=True, exist_ok=True) + + text = source.read_text(encoding="utf-8") + if source == (ROOT / "README.md").resolve(): + text = rewrite_markdown_links(text, source.parent, page) + page.write_text(text.rstrip() + "\n", encoding="utf-8") + return + + title = title_for(source) + text = normalize_markdown(text, source, title) + text = rewrite_markdown_links(text, source.parent, page) + text = rewrite_html_images(text, source.parent, page) + source_name = source.relative_to(ROOT).as_posix() + page.write_text( + f"# {title}\n\n" + f"{generated_toctree(source, page)}" + f"_Source: `{source_name}`_\n\n" + f"{text.rstrip()}\n", + encoding="utf-8", + ) + + +def write_example_index(directory: Path) -> None: + if (directory / "README.md").resolve() in EXAMPLE_READMES: + return + children = example_children(directory) + if not children: + return + + page = example_page(directory) + page.parent.mkdir(parents=True, exist_ok=True) + entries = [relative(example_page(child).with_suffix(""), page) for child in children] + lines = ["## Sections", ""] + for child in children: + child_link = f"[{child.name}]({relative(example_page(child), page)})" + grandchildren = [ + f"[{grandchild.name}]({relative(example_page(grandchild), page)})" + for grandchild in example_children(child) + ] + lines.append(f"- {child_link}: {', '.join(grandchildren)}" if grandchildren else f"- {child_link}") + + page.write_text( + f"# {'Examples' if directory == EXAMPLES else directory.name}\n\n" + f"{toctree(entries)}\n" + + "\n".join(lines) + + "\n", + encoding="utf-8", + ) + + +def main() -> None: + for directory in (GENERATED, GENERATED_MODELS, GENERATED_EXAMPLES): + if directory.exists(): + shutil.rmtree(directory) + directory.mkdir(parents=True, exist_ok=True) + + sources = [ + *PROJECT_PAGES, + COMMON_MATH, + PHASOR_INPUT_FORMAT, + *MODEL_READMES, + *EXAMPLE_READMES, + ] + for source in sources: + write_page(source) + for directory in sorted(EXAMPLE_DOC_DIRS): + write_example_index(directory) + + +if __name__ == "__main__": + main() diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 000000000..73347a2b5 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,17 @@ +# GridKit + +```{toctree} +:maxdepth: 4 +:titlesonly: +:hidden: + +generated/install +applications/index +models/index +examples/generated/index +api +development/index +``` +```{include} generated/readme.md +:relative-docs: generated/ +``` diff --git a/docs/models/index.md b/docs/models/index.md new file mode 100644 index 000000000..8ac0a888d --- /dev/null +++ b/docs/models/index.md @@ -0,0 +1,21 @@ +# Models + +```{toctree} +:maxdepth: 4 +:titlesonly: +:hidden: + +generated/common-math +generated/emt/index +generated/phasor-dynamics/index +generated/power-electronics/index +generated/power-flow/index +``` + +## Sections + +- [CommonMath](generated/common-math.md) +- [EMT](generated/emt/index.md) +- [PhasorDynamics](generated/phasor-dynamics/index.md) +- [PowerElectronics](generated/power-electronics/index.md) +- [PowerFlow](generated/power-flow/index.md) diff --git a/docs/requirements.txt b/docs/requirements.txt new file mode 100644 index 000000000..7f0c2b7ed --- /dev/null +++ b/docs/requirements.txt @@ -0,0 +1,5 @@ +sphinx +breathe +exhale +myst-parser +sphinx-rtd-theme diff --git a/examples/PhasorDynamics/Tiny/TwoBus/Gensal/README.md b/examples/PhasorDynamics/Tiny/TwoBus/Gensal/README.md index 475c30031..0d7cf94e2 100644 --- a/examples/PhasorDynamics/Tiny/TwoBus/Gensal/README.md +++ b/examples/PhasorDynamics/Tiny/TwoBus/Gensal/README.md @@ -9,8 +9,8 @@ monitored machine states are compared against `GENSAL.ref.csv`. ## Trajectory Comparison -![GENSAL validation trajectory](GENSAL.png) +![GENSAL validation trajectory](Gensal_validation.png) ## Error -![GENSAL validation error](GENSAL_ERROR.png) +![GENSAL validation error](Gensal_validation_error.png)