[Feature] Enable safe per-request LoRA loading (implementation included)

[MateusGPe]

Feature Summary

Support for safe, per-request LoRA loading when running in HTTP Server mode

Detailed Description

I would like to propose a patch to enable per-request LoRA loading when running in HTTP Server. Currently, the server ignores LoRA tags (e.g., lora:name:0.5) in prompts because the generation parameters are initialized with an empty string for the LoRA directory.

I am unsure of the original motivation for explicitly disabling this path, but simply enabling it causes "weight stacking" (state pollution) on the persistent server process.

The attached diff implements the following:

1. Enables Parsing: Updates gen_params.process_and_check to use ctx_params.lora_model_dir, allowing the backend to find and load the LoRA files.
2. Enables Safety: Forces ctx_params.lora_apply_mode = LORA_APPLY_AT_RUNTIME during server initialization. This ensures LoRA calculations are applied dynamically during graph execution without permanently altering the base model weights.

Proposed Change

Diff

diff --git a/examples/server/main.cpp b/examples/server/main.cpp
index 5c951c0..4fb57b0 100644
--- a/examples/server/main.cpp
+++ b/examples/server/main.cpp
@@ -282,6 +282,7 @@ int main(int argc, const char** argv) {
     LOG_DEBUG("%s", default_gen_params.to_string().c_str());
 
     sd_ctx_params_t sd_ctx_params = ctx_params.to_sd_ctx_params_t(false, false, false);
+    ctx_params.lora_apply_mode    = LORA_APPLY_AT_RUNTIME;
     sd_ctx_t* sd_ctx              = new_sd_ctx(&sd_ctx_params);
 
     if (sd_ctx == nullptr) {
@@ -392,7 +393,7 @@ int main(int argc, const char** argv) {
                 return;
             }
 
-            if (!gen_params.process_and_check(IMG_GEN, "")) {
+            if (!gen_params.process_and_check(IMG_GEN, ctx_params.lora_model_dir)) {
                 res.status = 400;
                 res.set_content(R"({"error":"invalid params"})", "application/json");
                 return;
@@ -570,7 +571,7 @@ int main(int argc, const char** argv) {
                 return;
             }
 
-            if (!gen_params.process_and_check(IMG_GEN, "")) {
+            if (!gen_params.process_and_check(IMG_GEN, ctx_params.lora_model_dir)) {
                 res.status = 400;
                 res.set_content(R"({"error":"invalid params"})", "application/json");
                 return;

Alternatives you considered

• Naive Implementation: Just passing the directory without changing the apply mode. This resulted in model corruption after a few requests due to weight accumulation.
• External Management: Running separate server instances for different LoRA configurations, which is resource-heavy and lacks flexibility.

Additional context

• File: examples/server/main.cpp
• Logic: common.hpp (Lines ~440) describes at_runtime as the method that avoids precision issues and permanent weight modification, making it suitable for a long-running server process.

[stduhpf]

I think setting the lora model dir in http server is disabled for security reasons. Letting remote users attempt to load data from arbitrary paths on your machine is no good.

[MateusGPe]

@stduhpf That is a valid concern regarding Directory Traversal. However, given that ctx_params.lora_model_dir is already strictly defined by the server admin at startup, I believe the correct approach is to sanitize the input rather than disable the feature entirely.
If we ensure that the parsed LoRA name cannot contain directory separators (/, \) or relative path components (..), the server is safe because it can only load files physically present in the admin-defined lora_model_dir.
I propose we add a simple sanitizer or a check that enforces the resolved path remains within lora_model_dir. Alternatively, we could gate this behavior behind a CLI flag (e.g., --http-enable-lora) so it is an opt-in feature for admins who need it.

[stduhpf]

Ah I misunderstood what you meant, I thought you were asking for setting the lora model dir via http request.

#830 should handle the sanitization of lora paths.

[MateusGPe]

@stduhpf I'm new to this, and I'm not aware of some implications in the context of sd.cpp. I'm only a hobbyist who messes around with C++, but I think that something like this solves the issue:

diff --git a/examples/common/common.hpp b/examples/common/common.hpp
index 868b06e..93d3e39 100644
--- a/examples/common/common.hpp
+++ b/examples/common/common.hpp
@@ -993,6 +993,55 @@ static bool is_absolute_path(const std::string& p) {
 #endif
 }
 
+static bool sanitize_lora_path(const std::string& lora_model_dir,
+                               const std::string& raw_path_str,
+                               fs::path& full_path) {
+    if (lora_model_dir.empty()) {
+        return false;
+    }
+
+    fs::path raw_path(raw_path_str);
+
+    // Disallow absolute paths.
+    if (raw_path.is_absolute()) {
+        LOG_WARN("lora path must be relative: %s", raw_path_str.c_str());
+        return false;
+    }
+
+    // Disallow paths that contain '..'
+    for (const auto& part : raw_path) {
+        if (part == "..") {
+            LOG_WARN("lora path cannot contain '..': %s", raw_path_str.c_str());
+            return false;
+        }
+    }
+
+    fs::path lora_dir(lora_model_dir);
+    full_path = lora_dir / raw_path;
+
+    // Canonicalize both paths to resolve any symlinks.
+    auto canonical_lora_dir  = fs::weakly_canonical(lora_dir);
+    auto canonical_full_path = fs::weakly_canonical(full_path);
+
+    // Check if the canonical path of the lora file is within the canonical lora directory.
+    std::string lora_dir_str  = canonical_lora_dir.string();
+    std::string full_path_str = canonical_full_path.string();
+
+    if (full_path_str.rfind(lora_dir_str, 0) != 0) {
+        LOG_WARN("lora path is outside of the lora model directory: %s", raw_path_str.c_str());
+        return false;
+    }
+
+    // Block access to subdirectories (optional? drawback: users cannot organize their LoRAs into subfolders.)
+    fs::path relative_path = canonical_full_path.lexically_relative(canonical_lora_dir);
+    if (relative_path.has_parent_path() && !relative_path.parent_path().empty()) {
+        LOG_WARN("lora path in subdirectories is not allowed: %s", raw_path_str.c_str());
+        return false;
+    }
+
+    return true;
+}
+
 struct SDGenerationParams {
     std::string prompt;
     std::string prompt_with_lora;  // for metadata record only
@@ -1606,10 +1655,10 @@ struct SDGenerationParams {
             }
 
             fs::path final_path;
-            if (is_absolute_path(raw_path)) {
-                final_path = raw_path;
-            } else {
-                final_path = fs::path(lora_model_dir) / raw_path;
+            if (!sanitize_lora_path(lora_model_dir, raw_path, final_path)) {
+                tmp    = m.suffix().str();
+                prompt = std::regex_replace(prompt, re, "", std::regex_constants::format_first_only);
+                continue;
             }
             if (!fs::exists(final_path)) {
                 bool found = false;
@@ -1617,7 +1666,7 @@ struct SDGenerationParams {
                     fs::path try_path = final_path;
                     try_path += ext;
                     if (fs::exists(try_path)) {
-                        final_path = try_path;
+                        final_path = try_path.lexically_normal();
                         found      = true;
                         break;
                     }

[MateusGPe]

I used this for a while and found an issue. It's possible to access the folder itself. This fixes the previous fix:

    auto canonical_lora_dir  = fs::weakly_canonical(lora_dir);
    auto canonical_full_path = fs::weakly_canonical(full_path);

    if (fs::is_directory(canonical_full_path)) {
        LOG_WARN("lora path resolved to a directory, not a file: %s", raw_path_str.c_str());
        return false;
    }