Run models in parallel during benchmark (#53)

* Run models in parallel during benchmark * Updating eval docs * Formatting * Unused context * Handling race conditions

Author: steven10a

docs/evals.md

@@ -67,6 +67,8 @@ This installs:
 |`--azure-api-version`| ❌ | Azure OpenAI API version (default: 2025-01-01-preview) |
 |`--models`| ❌ | Models for benchmark mode (benchmark only) |
 |`--latency-iterations`| ❌ | Latency test samples (default: 25) (benchmark only) |
+|`--max-parallel-models`| ❌ | Maximum number of models to benchmark concurrently (default: max(1, min(model_count, cpu_count))) (benchmark only) |
+|`--benchmark-chunk-size`| ❌ | Optional number of samples per chunk when benchmarking to limit long-running runs (benchmark only) |
 
 ## Configuration
 
@@ -205,6 +207,8 @@ guardrails-evals \
 -**Automatic stage detection**: Evaluates all stages found in configuration
 -**Batch processing**: Configurable parallel processing
 -**Benchmark mode**: Model performance comparison with ROC AUC, precision at recall thresholds
+-**Parallel benchmarking**: Run multiple models concurrently (defaults to CPU count)
+-**Benchmark chunking**: Process large datasets in chunks for better progress tracking
 -**Latency testing**: End-to-end guardrail performance measurement
 -**Visualization**: Automatic chart and graph generation
 -**Multi-provider support**: OpenAI, Azure OpenAI, Ollama, vLLM, and other OpenAI-compatible APIs