This project implements a FastAPI-based local server designed to load one or more pre-trained NLP models during startup and expose them through a clean, RESTful API for inference.
For example, it leverages the Hugging Face transformers library to load the CIRCL/vulnerability-severity-classification-distilbert-base-uncased model, which specializes in classifying vulnerability descriptions according to their severity level. The server initializes this model once at startup, ensuring minimal latency during inference requests.
Clients interact with the server via dedicated HTTP endpoints corresponding to each loaded model. Additionally, the server automatically generates comprehensive OpenAPI documentation that details the available endpoints, their expected input formats, and sample responses—making it easy to explore and integrate the services.
The ultimate goal is to enrich vulnerability data descriptions through the application of a suite of NLP models, providing direct benefits to Vulnerability-Lookup and supporting other related projects.
Seems fun like I could just plug it into openwebui docker compose. Mind you is the only benefit of this that u can run non ollama compatible models.