After running the humaneval-sythesize on a large number of samles, I noticed that many processes "a.out" and "node test.js" are runing, with runtimes of several hours and huge memory consumption. Appearently the evaluation does not kill processes after a set timeout.
Example command
accelerate launch --main_process_port 0 main.py --tasks humanevalsynthesize-cpp --load_generations_path ../.. --metric_output_path ../.. --allow_code_execution --n_samples 1000
After running the humaneval-sythesize on a large number of samles, I noticed that many processes "a.out" and "node test.js" are runing, with runtimes of several hours and huge memory consumption. Appearently the evaluation does not kill processes after a set timeout.
Example command