0%
May 6, 2025

Environment Variables via Secret Managers in Backend using Expo Approach

aws

env

secrets-manager

Contents

How Expo works with Environment Variables?

This is how EXPO works when using eas commands:

"scripts": {
    "env:pull:dev": "eas env:pull --environment development --non-interactive",
    "env:pull:uat": "eas env:pull --environment preview --non-interactive",
    "env:pull:prod": "eas env:pull --environment production --non-interactive"
}

It pulls the variable defined in expo account:

into local file called .env.local.

  • That means whenever we want to debug/develop in dev environement, we simply yarn env:pull:dev and start our task.

  • The same can be applied to backend and in the sequel we explain an approach applicable to both express and spring boot application.

Secret Managers

Define variables part I: manual approach

Simply create a secret and save the key-value pairs into it:

  • Note that our variable are deliberately defined by a.b.c because on one hand this is a standard definition in application.properties, and on the other hand there is no way to define nested key-value pair.

  • We will use nodejs script to download the secret as a json string.

  • We will also use lodash to rewrite that json string into a json object with nested properites. From that we are free to translate this nested json into yml or keep it as it is.

  • This approach is not ideal as we need to add and manage these variables in aws console, which is intractable when our projects grow, especially when these variables are shared to all projects.

    We will introduce type-safe approach in Define variables part II.

Objective

  • Take spring boot as an example. For developers our resources folder is very clean:

    we just have a globally shared variables (application.yml). We will pull application-local.yml for DEV, UAT and PROD respectively for development purpose

  • Yes, they have the same filename for all environments: application-local.yml, as is the pattern in expo service.

  • Next we will create an npm script so that when we run yarn env:pull:uat:

    we get the application-local.yml for UAT, which unifies the scripts in development and deployment.

    For spring we can further execute this npm script via gradle task, we will introduce it in the upcoming sections.

Nodejs scripts in package.json
Make sure you have correct credentials

Assume that we have configured an account using aws cli in our machine (if not, install the cli and execute aws configure, then our credentials will be stored in ~/.aws).

Assume also that we have appropriate policy to read-write value in secret managers, then we can extract the secret in the by the following:

package.json

Here we define a set of env:pull:{stage}.

{
  "license": "MIT",
  "scripts": {
    "env:pull:dev": "yarn && npx tsm download.ts --secret_name billie-backend-dev --format yml --save_at src/main/resources/application-local.yml",
    "env:pull:dev-internal": "yarn && npx tsm download.ts --secret_name billie-backend-dev-internal --format yml --save_at src/main/resources/application-local-internal.yml",
    "env:pull:uat": "yarn && npx tsm download.ts --secret_name billie-backend-uat --format yml --save_at src/main/resources/application-local.yml",
    "env:pull:uat-internal": "yarn && npx tsm download.ts --secret_name billie-backend-uat-internal --format yml --save_at src/main/resources/application-local-internal.yml",
    "env:pull:prod": "yarn && npx tsm download.ts --secret_name billie-backend-prod --format yml --save_at src/main/resources/application-local.yml",
    "env:pull:prod-internal": "yarn && npx tsm download.ts --secret_name billie-backend-prod-internal --format yml --save_at src/main/resources/application-local-internal.yml"
  },
  "devDependencies": {
    "tsm": "^2.3.0".
    ...
  },
}

We will explain the download.ts right below this sub-section.

Note that we have yarn add tsm which is a light-weight binary to execute typescript files without writing any tsconfig.json.

The internal variants are for lambda functions which use internal resource rather than publicly accessible resources and are simply for deployment purpose.

download.ts

I have written an npm package on pulling variables from a secret in aws secrets-manager into json | yml | flat_env format, let's

yarn add secrets-manager-to-config

then create a file called download.ts:

// download.ts

import { downloadConfig, SecretConfig } from "secrets-manager-to-config";

const secretConfig: SecretConfig = {
  awsRegion: "ap-southeast-2",
};

downloadConfig(secretConfig);

now rom secrets manager in the f

ts-node download.ts --secret_name <secret-name> \
  --format <format> \
  --save_at <filepath>

will download the secret secret-name from secrets manager in the format of json | yml | flat_env at filepath.

For spring boot, we define gradle tasks in build.gradle.kts
// build.gradle.kts

tasks.register<Exec>("dev") {
    description = "pull dev environment variables"
    group = "environement variables"
    configureNpmCommand(this, "env:pull:dev")
    doLast {
        handleCommandResult(this as Exec)
    }
}

tasks.register<Exec>("dev-internal") {
    description = "pull dev VPC internal variables"
    group = "environement variables"
    configureNpmCommand(this, "env:pull:dev-internal")
    doLast {
        handleCommandResult(this as Exec)
    }
}

tasks.register<Exec>("uat") {
    description = "pull uat environment variables"
    group = "environement variables"
    configureNpmCommand(this, "env:pull:uat")
    doLast {
        handleCommandResult(this as Exec)
    }
}

tasks.register<Exec>("uat-internal") {
    description = "pull uat VPC internal variables"
    group = "environement variables"
    configureNpmCommand(this, "env:pull:uat-internal")
    doLast {
        handleCommandResult(this as Exec)
    }
}

tasks.register<Exec>("prod") {
    description = "pull prod environment variables"
    group = "environement variables"
    configureNpmCommand(this, "env:pull:prod")
    doLast {
        handleCommandResult(this as Exec)
    }
}

tasks.register<Exec>("prod-internal") {
    description = "pull prod environment variables"
    group = "environement variables"
    configureNpmCommand(this, "env:pull:prod-internal")
    doLast {
        handleCommandResult(this as Exec)
    }
}


fun configureNpmCommand(exec: ExecSpec, npmScript: String) {
    // Use shell to execute command
    if (System.getProperty("os.name").toLowerCase().contains("windows")) {
        exec.commandLine("cmd", "/c", "yarn $npmScript")
    } else {
        // For macOS/Linux - use the full path to npm
        exec.commandLine("sh", "-c", "/usr/local/bin/yarn $npmScript")
    }

    // Continue even if there's an error
    exec.isIgnoreExitValue = true
}

// Function to handle the command result
fun handleCommandResult(execTask: Exec) {
    if (execTask.executionResult.get().exitValue == 0) {
        println("✅ Env variables pulled successfully")
    } else {
        println("❌ Failed to pull env variables. Exit code: ${execTask.executionResult.get().exitValue}")
    }
}

which gives us a list of nice command to execute:

Result: pull a yml file from secrets manager

After executing the gradle task to pull dev variables (or equivalently we run yarn env:pull:dev), we get:

Now execute the spring boot project with profile local, we have run the application with combined "environment variables".

Define variables part II: type-safe approach via config in ts-files

NPM package we use: secrets-manager-to-config

I will rely heavily on my own npm package, let's

yarn add secrets-manager-to-config
upload.ts: Upload the default export from ts-file as a secret

I have designed the following API:

ts-node upload.ts --secret_name some-test-config --ts_path config/test.ts

where

  • // upload.ts

import { SecretConfig, uploadConfig } from "secrets-manager-to-config";

const secretConfig: SecretConfig = {
  awsRegion: "ap-southeast-2",
};

uploadConfig(secretConfig);

  • and the config/test.ts:

    // config/test.ts

const someConfig: {
  hihi: string;
  ohMy: {
    value: string;
    someArray: string[];
  };
} = {
  hihi: "bye",
  ohMy: {
    value: "gosh",
    someArray: ["hihi", "hjaaaaaaa"],
  },
};

export default someConfig;
The uploaded result in secrets manager

The uploaded secret in secrets manager now becomes:

Note that we can convert this into nested json/yml format by download.ts, you may recall it from here.

Manage real projects across different environments
Define variables with type-safty

Now we simply define variables in DEV, and for UAT and PROD we infer the type from typeof devObject:

in this way there is no mismatch on the number of variables across different environments.

Sync all variables to secrets manager

Finally we run "sync": "ts-node index.ts" to upload all secrets.

// index.ts

import { exec, ExecException } from "child_process";

function runCommand(command: string): Promise<string> {
    return new Promise((resolve, reject) => {
        exec(command, (error: ExecException | null, stdout: string, stderr: string) => {
            if (error) {
                reject(`Error: ${error.message}`);
                return;
            }
            if (stderr) {
                reject(`stderr: ${stderr}`);
                return;
            }
            resolve(stdout);
        });
    });
}

const envFiles: { secretName: string, tsFilePath: string }[] = [
    {
        secretName: "some-project-dev",
        tsFilePath: "some-project/dev/dev.ts"
    },
    {
        secretName: "some-project-uat",
        tsFilePath: "some-project/uat/uat.ts"
    },
]

// Secrets manager is blocking, so promise.all has no advantage
async function main() {
    for (const env of envFiles) {
        const { secretName, tsFilePath } = env;
        const cmd = `ts-node upload.ts --secret_name ${secretName} --ts_path ${tsFilePath}`
        console.log("executing", cmd)
        await runCommand(cmd)
    }
}

main();