feat: add configurable log levels via env.logLevel

[taronsung]

Summary

Adds configurable logging levels to control library verbosity, as requested in #117.

Changes

• Add LogLevel enum with NONE, ERROR, WARNING, INFO, DEBUG levels
• Add env.logLevel configuration (defaults to WARNING)
• Create logger utility that respects the configured log level
• Update console.log/warn calls in pipelines.js to use logger
• Export LogLevel and logger from main entry point

Usage

import { env, LogLevel, logger } from '@huggingface/transformers';

// Set log level to suppress all logs
env.logLevel = LogLevel.NONE;

// Set log level to show only errors
env.logLevel = LogLevel.ERROR;

// Set log level to show warnings and above (default)
env.logLevel = LogLevel.WARNING;

// Set log level to show info messages and above
env.logLevel = LogLevel.INFO;

// Set log level to show all messages including debug
env.logLevel = LogLevel.DEBUG;

// Use the logger directly
logger.info('Model loaded successfully');
logger.warn('Deprecated method used');
logger.error('Failed to load model');
logger.debug('Token count:', tokens.length);

Notes

• This PR focuses on the infrastructure and updates pipelines.js as a starting point
• Other files with console.warn/error calls can be migrated in follow-up PRs
• The TextStreamer output in streamers.js was intentionally not changed as it's user-facing streaming output, not logging

Closes #117

[xenova]

Hi @taronsung 👋 Thanks for the PR! I do like this feature, and it's definitely something I think we can implement. Since we're in the process of a relatively large refactor (v4: https://github.com/huggingface/transformers.js/tree/v4), could you update the base to be that branch? From the looks of it, it'd just entail moving the new files into the packages/transformers folder... all else should stay mostly similar.

We could also take inspiration from the python transformers library... maybe implementing methods like warning_once (for example) to only log once in case of repeated calls. WDYT? Edit: Actually, maybe not worth it because browser runtimes typically collapse duplicate logs anyway.

[nico-martin]

I added some commits to the PR. Most of it is just more console. that I wanted to replace with logger., but I also added a mapping between our own logLevels and the ONNX_ENV.logLevel.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[xenova]

main comment is here; next, for some reason I'm unable to merge main into this branch. @taronsung could you rebase? alternatively, @nico-martin if you have permission, could you ensure the PR is on latest main? thanks!

[nico-martin]

@xenova, I merged origin/main into this branch so its now up to date with main. Imo its ready to be merged :D

[xenova]

We might as well use our own log level dict and then use LogLevel.WARNING here (instead of a magic number)

[xenova]

same here

[xenova]

Thanks for iterating!

[fs-eire]

Just saw this PR. I need to mention that ONNX_ENV.logLevel only takes effect when creating the OrtEnv, which is a singleton that lives forever. If set ONNX_ENV.logLevel to "warning" and create a session and then change it to "verbose", it won't work.

Also, if setting ONNX_ENV.logLevel to "warning" but setting logSeverityLevel in session options to 0 (verbose), it will not output any logs and will cause the inference become slower. The reason is logSeverityLevel setting to 0 will cause the log string to be created and push to the log sink (which is avoid if setting logSeverityLevel to 3), but the actual logger won't output the log according to ONNX_ENV.logLevel.

A little bit confusing but need to be careful for this part. @xenova

[xenova]

Thanks for the info @fs-eire 👍 Do you have any suggestions to improve this? Luckily it shouldn't affect most users, and the default we have chosen are much better than what they used to be (outputting far too many logs).

This is similar to a problem I described to @nico-martin, which makes session-level settings a bit more difficult to implement (since ORT uses global settings for many things, like proxying). Ref: #1479

[fs-eire]

based on the current limitation, we probably have something a "production" mode and a "diagnose" mode, where in production mode we can set ORT env log level to error, and in diagnose mode we set ORT env log level to verbose (a little bit slower but it's fine since it's diagnose mode). This should match most users' expectation.

[nico-martin]

It's not ideal, but I think it's quite rare for ORT variables to be changed by the user.
My main point with #1479 is not mecessarily about ORT variables, but about session specific variables. Like remoteHost, remotePathTemplate, etc. Those are things I am concerned because they are not linked to the environment, but rather to a specific session. Especially ehen the ecosystem grows and more dependencies use Transformers.js as a dependency. Then all of a sudden the user has no idea why installing the feature-extraction-package influences the simple-web-llm-package (one might set a different remoteHost).

TLDR: I can live with certain global dependencies that we cannot eliminate for good reasons. :D