Memory leak when file watcher enabled on Windows

[andriysavin]

Description

I'm observing enourmous memory consumption by neovim caused by nvim-tree plugin when using with large directory hiearchies like C: drive as root. My findings show that the cause of that is file system watcher. I know that the plugin is documented (aka by-design) to perform poorly in such scenarios, but I have some more thoughts on this.

1. I tried to do memory profiling of neovim in VS, and found out that most allocations occur in calls to luv_fs_event_cb which apparently puts lua callback request in a queue, and the queue is growing continuously.
2. Browsing nvim-tree source code and monitoring logs, I found out that nvim-tree registers FS watcher for every child directory and file separately when a directory is expanded. I saw the comment that recursive events monitoring is not working for some reason (though AFAIK on windows FS watcher API works ok with hierarchies). What I think this leads to is that with huge amount of callback regisrations (I counted almost 40 000 (!) in the log for just expanding c:\Users\MyUserName) neovim api has to poll for events (via ReadDirectoryChangesW system call) for each file system item separately. Results return happens asynchronously, so callback requests are queued in parallel from multiple threads, which increases chances of queue growing faster than it can be processed.
3. As mentioned in 2, the amount of watcher registrations does not correlate with amount of expanded direcories and visible children. In the log I see many records like "Watcher:create 'C:\Users\MyUserName\AppData\Local\Microsoft\VSApplicationInsights\vstelf'", while I just expanded "c:\Users\MyUserName". In nvim-tree code I see that children exploration goes just one level, but in some cases it goes deeper recursively.
   I don't quite understand why, by I believe this is the reason the number of watched items is so huge.
   Another clue I realized from the logs: watched directories are refreshed on events (via git calls?): [watcher] node event executing refresh. When this happens, it looks like children exploration and watch registration happens to directories which so far have been leaves themselves. That way, nvim-tree gradually continues digging folder hierarchy (and adding watchers) without folders even been expanded.

My questions/suggestions:

1. If I expand a folder, only immediate children of this folder should be monitored at worst.
2. If a folder is collapsed, it would be nice to stop watching its children recursively (e.g. if children were expanded as well). I see that with expanded children this might be tricky.
3. I know there is an option to exclude (blacklist) certain folders from watching. But it would extremely helpful in this scenario to be able to do the opposite: watch only included folders (whitelist). That is, watching is disabled systemwide so I can use nvim-tree for general file system browsing without being afraid it starts choking with file system watch events, but my projects directory is white-listed, so it's monitored once I start browsing it.

Neovim version

NVIM v0.11.6
Build type: Debug
LuaJIT 2.1.1741730670

Operating system and version

Windows 11 Pro

Windows variant

No response

nvim-tree version

e16cd38

Clean room replication

vim.g.loaded_netrw = 1
vim.g.loaded_netrwPlugin = 1

vim.cmd([[set runtimepath=$VIMRUNTIME]])
vim.cmd([[set packpath=/tmp/nvt-min/site]])
local package_root = "/tmp/nvt-min/site/pack"
local install_path = package_root .. "/packer/start/packer.nvim"
local function load_plugins()
  require("packer").startup({
    {
      "wbthomason/packer.nvim",
      "nvim-tree/nvim-tree.lua",
      "nvim-tree/nvim-web-devicons",
      -- ADD PLUGINS THAT ARE _NECESSARY_ FOR REPRODUCING THE ISSUE
    },
    config = {
      package_root = package_root,
      compile_path = install_path .. "/plugin/packer_compiled.lua",
      display = { non_interactive = true },
    },
  })
end
if vim.fn.isdirectory(install_path) == 0 then
  print("Installing nvim-tree and dependencies.")
  vim.fn.system({ "git", "clone", "--depth=1", "https://github.com/wbthomason/packer.nvim", install_path })
end
load_plugins()
require("packer").sync()
vim.cmd([[autocmd User PackerComplete ++once echo "Ready!" | lua setup()]])
vim.opt.termguicolors = true
vim.opt.cursorline = true

-- MODIFY NVIM-TREE SETTINGS THAT ARE _NECESSARY_ FOR REPRODUCING THE ISSUE
_G.setup = function()
---@type nvim_tree.config
  local config = {
    disable_netrw = true,
    hijack_cursor = true,
    sync_root_with_cwd = true,
    update_focused_file = {
        enable = true,
        update_root = false,
    }
  }
  require("nvim-tree").setup(config)
end

-- UNCOMMENT this block for diagnostics issues, substituting pattern and cmd as appropriate.
-- Requires diagnostics.enable = true in setup.
--[[
vim.api.nvim_create_autocmd("FileType", {
  pattern = "lua",
  callback = function()
    vim.lsp.start {
      name = "my-luals",
      cmd = { "lua-language-server" },
      root_dir = vim.loop.cwd(),
    }
  end,
})
]]

Steps to reproduce

1. c:;cd c:\
2. nvim.exe -nu path-to\nvt-min.lua
3. :NvimTreeOpen
4. Start expanding the tree: open Users, then your user home folder. No need to go deeper.
5. Possibly do some other activities on the system to trigger file system events (e.g. build something in VS, browse with the browser etc.)

Expected behavior

I expect nvim-tree not ro consume tons of memory just because of expanding couple of folders.

Actual behavior

Nvim-tree consumes huge amount of memory which keeps growing indefinitely.