Manage Environment Variables with Pydantic

Introduction

Builders work on functions which might be imagined to be deployed on some server with the intention to enable anybody to make use of these. Sometimes within the machine the place these apps dwell, builders arrange surroundings variables that enable the app to run. These variables might be API keys of exterior providers, URL of your database and way more.

For native growth although, it’s actually inconvenient to declare these variables on the machine as a result of it’s a gradual and messy course of. So I’d prefer to share on this brief tutorial learn how to use Pydantic to deal with surroundings variables in a safe manner.

.env file

What you generally do in a Python challenge is to retailer all of your surroundings variables in a file named .env. It is a textual content file containing all of the variables in a key : worth format. You need to use additionally the worth of one of many variables to declare one of many different variables by leveraging the {} syntax.

The next is an instance:

#.env file

OPENAI_API_KEY="sk-your personal key"
OPENAI_MODEL_ID="gpt-4o-mini"

# Improvement settings
DOMAIN=instance.org
ADMIN_EMAIL=admin@${DOMAIN}

WANDB_API_KEY="your-private-key"
WANDB_PROJECT="myproject"
WANDB_ENTITY="my-entity"

SERPAPI_KEY= "your-api-key"
PERPLEXITY_TOKEN = "your-api-token"

Remember the .env file ought to stay personal, so it is necessary that this file is talked about in your .gitignore file, to ensure that you by no means push it on GitHub, in any other case, different builders may steal your keys and use the instruments you’ve paid for.

env.instance file

To ease the lifetime of builders who will clone your repository, you could possibly embrace an env.instance file in your challenge. It is a file containing solely the keys of what’s supposed to enter the .env file. On this manner, different individuals know what APIs, tokens, or secrets and techniques on the whole they should set to make the scripts work.

#env.instance

OPENAI_API_KEY=""
OPENAI_MODEL_ID=""

DOMAIN=""
ADMIN_EMAIL=""

WANDB_API_KEY=""
WANDB_PROJECT=""
WANDB_ENTITY=""

SERPAPI_KEY= ""
PERPLEXITY_TOKEN = ""

python-dotenv is the library you employ to load the variables declared into the .env file. To put in this library:

pip set up python-dotenv

Now you should use the load_dotenv to load the variables. Then get a reference to those variables with the os module.

import os
from dotenv import load_dotenv

load_dotenv()

OPENAI_API_KEY = os.getenv('OPENAI_API_KEY')
OPENAI_MODEL_ID = os.getenv('OPENAI_MODEL_ID')

This methodology will first look into your .env file to load the variables you’ve declared there. If this file doesn’t exist, the variable will probably be taken from the host machine. Which means you should use the .env file on your native growth however then when the code is deployed to a bunch surroundings like a digital machine or Docker container we’re going to straight use the surroundings variables outlined within the host surroundings.

Pydantic

Pydantic is likely one of the most used libraries in Python for information validation. It’s also used for serializing and deserializing courses into JSON and again. It robotically generates JSON schema, lowering the necessity for handbook schema administration. It additionally offers built-in information validation, making certain that the serialized information adheres to the anticipated format. Lastly, it simply integrates with standard net frameworks like FastAPI.

pydantic-settings is a Pydantic characteristic wanted to load and validate settings or config courses from surroundings variables.

!pip set up pydantic-settings

We’re going to create a category named Settings. This class will inherit BaseSettings. This makes the default behaviours of figuring out the values of any fields to be learn from the .env file. If no var is discovered within the .env file it will likely be used the default worth if offered.

from pydantic_settings import BaseSettings, SettingsConfigDict

from pydantic import (
    AliasChoices,
    Subject,
    RedisDsn,
)


class Settings(BaseSettings):
    auth_key: str = Subject(validation_alias="my_auth_key")  
    api_key: str = Subject(alias="my_api_key")  

    redis_dsn: RedisDsn = Subject(
        'redis://consumer:move@localhost:6379/1', #default worth
        validation_alias=AliasChoices('service_redis_dsn', 'redis_url'),  
    )

    model_config = SettingsConfigDict(env_prefix='my_prefix_')

Within the Settings class above we’ve got outlined a number of fields. The Subject class is used to supply extra information about an attribute.

In our case, we setup a validation_alias. So the variable title to search for within the .env file is overridden. Within the case reported above, the surroundings variable my_auth_key will probably be learn as an alternative of auth_key.

You may as well have a number of aliases to search for within the .env file that you could specify by leveraging AliasChoises(choise1, choise2).

The final attribute model_config , incorporates all of the variables concerning a specific matter (e.g connection to a db). And this variable will retailer all .env var that begin with the prefix env_prefix.

Instantiate and use settings

The subsequent step can be to really instantiate and use these settings in your Python challenge.

from pydantic_settings import BaseSettings, SettingsConfigDict

from pydantic import (
    AliasChoices,
    Subject,
    RedisDsn,
)


class Settings(BaseSettings):
    auth_key: str = Subject(validation_alias="my_auth_key")  
    api_key: str = Subject(alias="my_api_key")  

    redis_dsn: RedisDsn = Subject(
        'redis://consumer:move@localhost:6379/1', #default worth
        validation_alias=AliasChoices('service_redis_dsn', 'redis_url'),  
    )

    model_config = SettingsConfigDict(env_prefix='my_prefix_')

# create instantly a settings object
settings = Settings()

Now what use the settings in different elements of our codebase.

from Settings import settings

print(settings.auth_key) 

You lastly have an easy accessibility to your settings, and Pydantic helps you validate that the secrets and techniques have the right format. For extra superior validation suggestions confer with the Pydantic documentation: https://docs.pydantic.dev/latest/

Ultimate ideas

Managing the configuration of a challenge is a boring however essential a part of software program growth. Secrets and techniques like API keys, db connections are what often energy your software. Naively you may hardcode these variables in your code and it’ll nonetheless work, however for apparent causes, this might not be a great follow. On this article, I confirmed you an introduction on learn how to use pydantic settings to have a structured and secure method to deal with your configurations.

💼 Linkedin ️| 🐦 X (Twitter) | 💻 Website