Skip to content

hyperpolymath/filesoup

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

89 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

FSLint: File System Intelligence & Linting

FSLint is a high-performance, cross-platform file system linter designed to make filesystems intelligible and navigable for real people. It applies tiny, composable plugins to files and directories, transforming scattered directory trees into logical, metadata-rich bundles — bringing the cohesion of macOS .app packages to Linux and Windows.

Core Concept

Modern operating systems scatter files across deep, opaque hierarchies. Configuration lives in one place, data in another, caches in a third. FSLint addresses this "file soup" by linting directory structures and surfacing contextual metadata about every file and folder it encounters.

Contextual metadata

Instant insights into git status, file age, duplicates, AI-generated content, exposed secrets, and more.

Bundle-fication

Logic to identify and treat complex multi-file directories as cohesive single entities (like macOS .app bundles), so users see one logical "thing" rather than a tree of fragments.

Query engine

A CLI-first filtering language for slicing through results by filename, extension, plugin tags, size thresholds, and plugin status.

Built-in Plugins

FSLint ships with a set of core plugins for immediate filesystem visibility. Each can be enabled or disabled individually.

Plugin Default Description

git-status

Displays branch information and uncommitted changes for git-tracked files.

file-age

Flags files modified within configurable thresholds (default: 7 days).

grouping

Categorises files into types: Media, Dependencies, Source, Config, etc.

bundle-check

Verifies whether a directory meets the criteria for a cohesive "Package."

version-detection

Finds versioned files (report_v1, file_final, backup_2).

ocr-status

Detects PDF text layers and OCR state.

ai-detection

Identifies AI-generated content via EXIF metadata and heuristics.

duplicate-finder

Hash-based duplicate detection across directories and volumes.

secret-scanner

Scans for exposed API keys, credentials, and high-entropy strings.

Quick Start

Installation

git clone https://github.com/hyperpolymath/file-soup.git
cd file-soup
cargo build --release
# Binary at ./target/release/fslint

Basic Usage

# Scan current directory with default plugins
fslint scan

# Output results as JSON
fslint scan --format json

# Scan a specific directory
fslint scan /path/to/project

# Simple output format (path + tags only)
fslint scan --format simple

Querying

# Find all Rust source files with uncommitted changes
fslint query "ext:rs git-status:Modified"

# Find the newest config file
fslint query "name:config ext:toml newest:true"

# Find large files tagged as media
fslint query "tag:media size_gt:1048576"

# Find exposed secrets
fslint enable secret-scanner
fslint query "secret-scanner:Alert"

Plugin Management

# List all plugins and their status
fslint plugins

# Enable a disabled plugin
fslint enable ai-detection

# Disable a plugin
fslint disable grouping

# View current configuration
fslint config

Query Language

FSLint supports a space-separated filter syntax. Combine filters freely:

Filter Example Description

name:<string>

name:config

Match filename substring.

ext:<string>

ext:rs

Match file extension.

newest:true

newest:true

Return only the most recent match.

tag:<string>

tag:media

Match plugin-assigned tags.

size_gt:<bytes>

size_gt:1048576

Files larger than threshold.

size_lt:<bytes>

size_lt:1024

Files smaller than threshold.

<plugin>:<status>

git-status:Modified

Match specific plugin result.

fslint query "name:report ext:pdf newest:true"

Architecture

The project is structured as a Rust workspace with discrete crates separating the plugin interface from the scanning engine and CLI.

file-soup/
├── fslint-plugin-api/       # Plugin trait definitions and core types
├── fslint-plugin-sdk/       # Helper utilities for plugin authors
├── fslint-core/             # Scanning engine, query parser, caching  (planned)
├── fslint-cli/              # CLI entry point and output formatting   (planned)
└── plugins/                 # Individual plugin crate modules         (planned)
    ├── git-status/
    ├── file-age/
    ├── grouping/
    ├── bundle-check/
    └── ...

Current Implementation Status

  • fslint-plugin-api — Complete. Defines the Plugin trait, PluginContext, PluginResult, PluginStatus, PluginError, and PluginMetadata types. Includes builder methods and unit tests.

  • fslint-plugin-sdk — Substantially complete. Provides path helpers, file metadata utilities (age, size formatting), regex-based pattern matching for common file categories, and context helpers. Includes tests and fuzzing.

  • fslint-core — Planned. Will contain the scanning engine with smart caching (keyed on path, mtime, size), .gitignore support, configurable max-depth, and the query parser.

  • fslint-cli — Planned. Will provide the fslint command with scan, query, plugins, enable, disable, and config subcommands. Multiple output formats: Table (default), JSON, Simple.

  • plugins/ — Planned. Each plugin as its own crate, depending on fslint-plugin-api and optionally fslint-plugin-sdk.

Output Formats

Table Format (Default)

File                               Git        Age        Group              Other
─────────────────────────────────────────────────────────────────────────────────
src/main.rs                        Modified   Today      -                  -
package.json                       Clean      This week  -                  -
node_modules/react/index.js        -          -          Node Dependencies  -

JSON Format

[
  {
    "path": "/project/src/main.rs",
    "size": 1234,
    "results": [
      {
        "plugin_name": "git-status",
        "status": "Alert",
        "message": "Modified",
        "color": "yellow"
      }
    ]
  }
]

Simple Format

src/main.rs [Modified, Today]
package.json [Clean, This week]
node_modules/react/index.js [Node Dependencies]

Configuration

FSLint stores configuration at ~/.config/fslint/config.toml:

enabled_plugins = ["git-status", "file-age", "grouping", "bundle-check"]

[scanner]
max_depth = 10
include_hidden = false
follow_symlinks = false
respect_gitignore = true

[plugin_config.file-age]
threshold_days = 7

[plugin_config.duplicate-finder]
min_size = 1024

[plugin_config.secret-scanner]
max_file_size = 1048576

Performance

  • Smart caching: Results cached by (path, mtime, size) — unchanged files reuse cache on re-scan.

  • Max depth: Default limit of 10 levels prevents deep recursion into node_modules and similar pits.

  • .gitignore support: Respects .gitignore by default, skipping irrelevant files.

  • Lazy execution: Only enabled plugins run.

Plugin Development

Create a new plugin in three steps:

  1. Create the plugin crate:

    mkdir plugins/my-plugin
    cd plugins/my-plugin
    cargo init --lib
  2. Add dependencies to Cargo.toml:

    [dependencies]
    fslint-plugin-api = { path = "../../fslint-plugin-api" }
    fslint-plugin-sdk = { path = "../../fslint-plugin-sdk" }
  3. Implement the Plugin trait in src/lib.rs:

    use fslint_plugin_api::{Plugin, PluginContext, PluginResult, PluginMetadata, PluginError};
    
    pub struct MyPlugin;
    
    impl Plugin for MyPlugin {
        fn metadata(&self) -> PluginMetadata {
            PluginMetadata {
                name: "my-plugin".into(),
                version: "0.1.0".into(),
                description: "My custom file intelligence plugin".into(),
                author: Some("Your Name".into()),
                enabled_by_default: false,
            }
        }
    
        fn check(&self, context: &PluginContext) -> Result<PluginResult, PluginError> {
            // Your plugin logic here
            Ok(PluginResult::active("my-plugin", "Found something interesting"))
        }
    }

See existing plugins for complete examples.

Roadmap

v0.2.0 — Core Engine

  • ❏ Scanning engine with caching (fslint-core)

  • ❏ CLI with scan, query, plugins subcommands (fslint-cli)

  • ❏ Parallel file scanning

  • ❏ Core plugins: git-status, file-age, grouping, bundle-check

v0.3.0 — Extended Intelligence

  • ❏ WASM-based plugin runtime for cross-platform extensions

  • ❏ Plugins: secret-scanner, ai-detection, duplicate-finder

  • ❏ macOS bundle collapsing (.app as single entity)

  • ❏ Plugin configuration via TUI

v1.0.0 — Stable

  • ❏ Shell extension integration (Nautilus, Dolphin, Finder, Explorer)

  • ❏ Shadow navigation for symlinks

  • ❏ Virtual filesystem across disks and cloud storage

  • ❏ Malware scanner, licence detector, dependency analyser plugins

Licence

MPL-2.0 (Palimpsest Licence)

Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) <j.d.a.jewell@open.ac.uk>

About

File management and analysis tools

Resources

License

Code of conduct

Contributing

Security policy

Stars

Watchers

Forks

Releases

No releases published

Sponsor this project

  •  

Packages

 
 
 

Contributors