Add my personal TF utility code
This commit is contained in:
parent
f76a3162ce
commit
111b454307
|
@ -0,0 +1,133 @@
|
||||||
|
# Terraform Utility scripts
|
||||||
|
|
||||||
|
The scripts in this directory were made to augment SZs development and
|
||||||
|
deployment of it's terraform plans.
|
||||||
|
|
||||||
|
They all rely on scaffolding provided by `direnv` and the hirearchy of
|
||||||
|
`.envrc` files in the repo which set context relevant environment
|
||||||
|
variables which the scripts here interact with to create a cohesive and
|
||||||
|
streamlined workflow.
|
||||||
|
|
||||||
|
## Typical workflow
|
||||||
|
|
||||||
|
Just like working with 'vanilla' terraform, the process follows:
|
||||||
|
|
||||||
|
```plaintext
|
||||||
|
+----------------------+-------------------------+
|
||||||
|
V | [mostly non-production] |
|
||||||
|
init -> plan -> apply -> | plan-destroy -> apply |
|
||||||
|
^ | +-------------------------+
|
||||||
|
+-------+
|
||||||
|
```
|
||||||
|
|
||||||
|
With some subtle differences and defaults which are sensible to SZ's
|
||||||
|
process, along with persistent logging.
|
||||||
|
|
||||||
|
## SZ's workflow
|
||||||
|
|
||||||
|
1. `init` - the first step:
|
||||||
|
* Initializes logging timestamp (aka `tf0`)
|
||||||
|
* Preprocess template files (all _sz.*.sz_tmpl files)
|
||||||
|
* Run `terraform init`
|
||||||
|
* Log output into `_logs` sub-directory.
|
||||||
|
2. `plan` - The 'brain':
|
||||||
|
* Run `terraform plan` with support for `TF_TARGET`
|
||||||
|
and `TF_TARGET_DESTORY` environment variable.
|
||||||
|
* Generate a `tfplan` file for `apply` action.
|
||||||
|
* Log output into `_logs` sub-directory.
|
||||||
|
3. `apply` - The heavy lifting:
|
||||||
|
* Always reads the `tfplan` file.
|
||||||
|
* Used for either _building_ or _destroying_.
|
||||||
|
* Log output into `_logs` sub-directory.
|
||||||
|
4. `plan-destroy` - Sensible destory planning:
|
||||||
|
* Initializes logging timestamp (aka `tf0`)
|
||||||
|
* Always layout the plan
|
||||||
|
* Always logged
|
||||||
|
* Supports `TF_TARGET_DESTROY`
|
||||||
|
|
||||||
|
## Concepts
|
||||||
|
|
||||||
|
* `direnv`'s `.envrc` sets-up the following envrionment variables:
|
||||||
|
* SZ_TF_NAME
|
||||||
|
* SZ_GCP_PROJECT
|
||||||
|
* SZ_GCP_PROJECT_ID
|
||||||
|
* TF_LOG, TF_LOG_PATH
|
||||||
|
* TF_VAR_PLAN_PATH="_.tmp.${SZ_TF_NAME}.tfplan"
|
||||||
|
* TF_VAR_OUT="-out ${TF_VAR_PLAN_PATH}"
|
||||||
|
* TF_VAR_FILE_CLI="-var-file="
|
||||||
|
* TF_VAR_FILE_CLI=""
|
||||||
|
|
||||||
|
* `tf-init` pre-processes template files `_sz.*.sz_tmpl`.
|
||||||
|
it is required first and foremost by `_sz.init.tf.sz_tmpl` which
|
||||||
|
generates the platform initialization code based on the environment
|
||||||
|
set by `direnv` - making sure you are always working with correct
|
||||||
|
gcloud environment.
|
||||||
|
All processed files will have the name pattern `_.gen.*`. Please note
|
||||||
|
that any file begining with `_.` (that's an underscore and a dot) are
|
||||||
|
ignored via `.gitignore`
|
||||||
|
|
||||||
|
* All planning is written to a persistent plan
|
||||||
|
(`_.tmp.<tf-plan-name>.tfplan`), whether it's deploying changes, new
|
||||||
|
components or destroying, the tfplan must be generated, along with the
|
||||||
|
log files.
|
||||||
|
|
||||||
|
* Logging - Each tf call is logged in 3 parallel locations:
|
||||||
|
* `_logs/0_0_lastrun.log` - the last `tf` run.
|
||||||
|
* `_logs/0_<action>` - the latest run of a terraform action (plan,
|
||||||
|
init, etc...). The rationale here is that all actions of a complete
|
||||||
|
workflow will be easily accessible regardless of timestamps.
|
||||||
|
* `_logs/yyyymmddHHmmss_<action>` - same as above, only timestamped.
|
||||||
|
This allows grouping operations that ran together in sequence,
|
||||||
|
breaking out to a separate sequences (by `tf0`) that will not
|
||||||
|
overwrite previous runs.
|
||||||
|
|
||||||
|
* `less-tf` - less with sensible defaults to review latest `tf` run
|
||||||
|
results. If no logs file is specifcied, `_logs/0_0_lastrun.log` will
|
||||||
|
be opened.
|
||||||
|
|
||||||
|
## Reference of Scripts
|
||||||
|
|
||||||
|
* `tf`:
|
||||||
|
executes `terraform` while preseving logs including ANSI coloring.
|
||||||
|
|
||||||
|
* `tf0`:
|
||||||
|
same as `tf`, however it will reset the logging timestamp.
|
||||||
|
|
||||||
|
* `tf-init`:
|
||||||
|
perform `init` step.
|
||||||
|
|
||||||
|
* `tf-plan` or `tfp` or `tf0-plan`
|
||||||
|
perform `plan` step. `tf0-plan` resets timestamp.
|
||||||
|
|
||||||
|
* `tf-plan-destroy` or `tfpd`
|
||||||
|
Perform `plan` step for destruction.
|
||||||
|
|
||||||
|
* `tf-apply` or `tfa`:
|
||||||
|
Apply after `plan` step. Unlike `terraform plan` this will not stop to
|
||||||
|
ask for permission, it is based on the preserved planned-state
|
||||||
|
supplied by `tf-plan` (above).
|
||||||
|
|
||||||
|
* `tf-extract`:
|
||||||
|
Extracts current state as a json of pairs of names and ids. For use
|
||||||
|
with `tf-import`.
|
||||||
|
|
||||||
|
* `tf-import`:
|
||||||
|
Given a json files of name/id pairs, generate an import script for
|
||||||
|
the existing state. This can be used as a non-binary state file.
|
||||||
|
An example to create such a script from the latest succesful `tf-apply`:
|
||||||
|
|
||||||
|
> ```bash
|
||||||
|
> tf-import _logs/0_9_last_state_ids.json > import-state.sh
|
||||||
|
> ```
|
||||||
|
|
||||||
|
## General utility scripts
|
||||||
|
|
||||||
|
Documentaion still pending for the following:
|
||||||
|
|
||||||
|
* `switch-dbg-tf-on`
|
||||||
|
* `less-tf`
|
||||||
|
* `clear-tf-env`
|
||||||
|
* `clear-tf-env-targets`
|
||||||
|
* `clear-tf-end-vars`
|
||||||
|
* `get-tf-env`
|
||||||
|
* `get-tf-env-plan`
|
|
@ -0,0 +1,57 @@
|
||||||
|
#!/usr/bin/env -S bash -c 'echo "Not a user script. source(aka .) only"'
|
||||||
|
|
||||||
|
# the correct way to load this file is to source it like this:
|
||||||
|
# eval "$(. _tf_aux_functions)"
|
||||||
|
|
||||||
|
function _tfSetLogTS() {
|
||||||
|
TF_LOG_TS=$(date -d "today" +"%Y%m%d%H%M%S")
|
||||||
|
export TF_LOG_TS
|
||||||
|
}
|
||||||
|
|
||||||
|
function _tf_sedFullStop() {
|
||||||
|
sed --unbuffered '/^===FULLSTOP===$/q' | sed --unbuffered '/^===FULLSTOP===$/d'
|
||||||
|
}
|
||||||
|
|
||||||
|
function _tf_save_exitCode() {
|
||||||
|
echo TF_EXITCODE="$1" > /tmp/TF_EXITCODE
|
||||||
|
}
|
||||||
|
|
||||||
|
# shellcheck disable=SC2120
|
||||||
|
function _tf_get_exit_code() {
|
||||||
|
unset TF_EXITCODE
|
||||||
|
if [[ -r /tmp/TF_EXITCODE ]]; then
|
||||||
|
source /tmp/TF_EXITCODE
|
||||||
|
fi
|
||||||
|
if [[ -z "$TF_EXITCODE" ]]; then
|
||||||
|
TF_EXITCODE=0
|
||||||
|
fi
|
||||||
|
if [[ "$1" != '-' ]]; then
|
||||||
|
echo "TF_EXITCODE=${TF_EXITCODE}"
|
||||||
|
fi
|
||||||
|
if [[ "$TF_EXITCODE" -ne 0 ]]; then
|
||||||
|
return "$TF_EXITCODE"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
function _tf_exit_code() {
|
||||||
|
# shellcheck disable=SC2016
|
||||||
|
_tf_get_exit_code || echo "return $TF_EXITCODE 2>/dev/null || exit $TF_EXITCODE"
|
||||||
|
}
|
||||||
|
|
||||||
|
[[ -e /tmp/TF_EXITCODE ]] && rm /tmp/TF_EXITCODE
|
||||||
|
unset TF_EXITCODE
|
||||||
|
|
||||||
|
function safe_load() {
|
||||||
|
if [[ -z "$(find -mindepth 1 -maxdepth 1 -type f -name "*.tf" -or -name "*.tf.sz_tmpl")" ]]; then
|
||||||
|
local TF_EXIST=''
|
||||||
|
[[ -d '_tf' ]] && TF_EXIST=" Did you forget to cd into _tf?"
|
||||||
|
>&2 printf "ERROR: No Terraform files found.%s\n" "$TF_EXIST"
|
||||||
|
return 2
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
[[ 1 -ne "$_TF_AUX_FUNCTIONS_LOADED" ]] \
|
||||||
|
&& printf "%s\n" \
|
||||||
|
"_TF_AUX_FUNCTIONS_LOADED=1" \
|
||||||
|
"source $(command -v _tf_aux_functions) || { X=$?; return $X 2>/dev/null || exit $X; }" \
|
||||||
|
|| safe_load
|
|
@ -0,0 +1,14 @@
|
||||||
|
#! /usr/bin/env bash
|
||||||
|
function _clear-tf-env() {
|
||||||
|
local PATTERN="${1}"
|
||||||
|
local TOCLEAR=$(get-tf-env "${PATTERN}")
|
||||||
|
[[ -z "$TOCLEAR" ]] && {
|
||||||
|
echo "Could not find environment variables matching: $(echo "^TF_${PATTERN}")"
|
||||||
|
} || {
|
||||||
|
unset $TOCLEAR
|
||||||
|
echo "Cleared the following vars: $(echo "$TOCLEAR" | xargs echo)"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
_clear-tf-env "${@}"
|
||||||
|
unset _clear-tf-env
|
|
@ -0,0 +1,7 @@
|
||||||
|
#! /usr/bin/env bash
|
||||||
|
function _clear-tf-env-targets() {
|
||||||
|
clear-tf-env '[A-Z_]*TARGET$'
|
||||||
|
}
|
||||||
|
|
||||||
|
_clear-tf-env-targets "${@}"
|
||||||
|
unset _clear-tf-env-targets
|
|
@ -0,0 +1,15 @@
|
||||||
|
#! /usr/bin/env bash
|
||||||
|
|
||||||
|
function _clear-tf-env() {
|
||||||
|
local PATTERN="${1}"
|
||||||
|
local TOCLEAR=$(get-tf-env "${PATTERN}")
|
||||||
|
[[ -z "$TOCLEAR" ]] && {
|
||||||
|
echo "Could not find environment variables matching: $(echo "^TF_${PATTERN}")"
|
||||||
|
} || {
|
||||||
|
unset $TOCLEAR
|
||||||
|
echo "Cleared the following vars: $(echo "$TOCLEAR" | xargs echo)"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
_clear-tf-env "${@}"
|
||||||
|
unset _clear-tf-env
|
|
@ -0,0 +1,12 @@
|
||||||
|
#! /usr/bin/env bash
|
||||||
|
|
||||||
|
function _get-tf-env() {
|
||||||
|
local PATTERN="${1-}"
|
||||||
|
compgen -v | grep "^TF_${PATTERN}" \
|
||||||
|
| while read a; do
|
||||||
|
set | grep "^$a=" | grep --color=auto '\b='
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
_get-tf-env "${@}"
|
||||||
|
unset _get-tf-env
|
|
@ -0,0 +1,17 @@
|
||||||
|
#! /usr/bin/env bash
|
||||||
|
|
||||||
|
function _get-tf-env-plan() {
|
||||||
|
#set | grep '^TF\(_\(VAR_[a-z]\)\|\([A-Z_]*TARGET=\)\)' | grep '\b='
|
||||||
|
|
||||||
|
local PATTERN="${1:-.*}"
|
||||||
|
compgen -v | grep "^TF\(_\(VAR_[a-z]\)\|\([A-Z_]*TARGET=\)\)" \
|
||||||
|
| while read a; do
|
||||||
|
set \
|
||||||
|
| grep "^$a=" \
|
||||||
|
| grep "${PATTERN}" \
|
||||||
|
| grep '\b=' --color
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
_get-tf-env-plan "${@}"
|
||||||
|
unset _get-tf-env-plan
|
|
@ -0,0 +1,35 @@
|
||||||
|
#! /usr/bin/env bash
|
||||||
|
eval "$(. _tf_aux_functions)"
|
||||||
|
|
||||||
|
function _less-tf() {
|
||||||
|
local _TF_LOG_FILE=${1-_logs/0_0_lastrun.log}
|
||||||
|
[[ "$1" == "-" ]] && _TF_LOG_FILE='_logs/0_0_lastrun.log'
|
||||||
|
[[ -z "$SZ_DEBUG" ]] || echo "lessts: ${@} | LESS_NO_WAIT = '${LESS_NO_WAIT}'"
|
||||||
|
[[ "$1" == "-" ]] && {
|
||||||
|
[[ -z "$SZ_DEBUG" ]] || echo "tailing..."
|
||||||
|
trap : INT;
|
||||||
|
tail -f $_TF_LOG_FILE | _tf_sedFullStop;
|
||||||
|
} \
|
||||||
|
|| { [[ -z "${LESS_NO_WAIT}" ]] \
|
||||||
|
&& {
|
||||||
|
[[ -z "$SZ_DEBUG" ]] || echo "Invoking less..."
|
||||||
|
less \
|
||||||
|
--no-init \
|
||||||
|
--raw-control-chars \
|
||||||
|
--line-numbers \
|
||||||
|
--quiet \
|
||||||
|
--hilite-unread \
|
||||||
|
--incsearch \
|
||||||
|
--ignore-case \
|
||||||
|
--force \
|
||||||
|
"${@:2}" -- \
|
||||||
|
"$_TF_LOG_FILE"
|
||||||
|
#\
|
||||||
|
#'+G?([^\w\W][\[0-9m+])Plan:' \
|
||||||
|
#'+3k'
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
_less-tf "${@}"
|
||||||
|
unset _less-tf
|
|
@ -0,0 +1,99 @@
|
||||||
|
#!/usr/bin/env -S bash -c 'echo "Not a user script. source(aka .) only"'
|
||||||
|
set -e
|
||||||
|
|
||||||
|
#
|
||||||
|
# Loads common environmetn for direnv enabled repos
|
||||||
|
function has() {
|
||||||
|
type "$1" &>/dev/null
|
||||||
|
}
|
||||||
|
|
||||||
|
get_clean_path() {
|
||||||
|
# shellcheck disable=SC2016
|
||||||
|
sed 's/ *:\?$//g;s/`/``/g;s/:/`:`/g;s/\\`:`/\\:/g;' <<< "$1" | \
|
||||||
|
awk -v RS='`:`' -v ORS='`:`' '!arr[$0]++' | \
|
||||||
|
sed 's/`:`/:/g;s/:$//g'
|
||||||
|
}
|
||||||
|
|
||||||
|
function PATH_add() {
|
||||||
|
for new_path in "${@}"; do
|
||||||
|
PATH="${new_path}:$PATH"
|
||||||
|
done
|
||||||
|
|
||||||
|
PATH=$( get_clean_path "$PATH" )
|
||||||
|
}
|
||||||
|
|
||||||
|
export LESS='--quit-if-one-screen --ignore-case --line-numbers --quiet --raw-control-chars --hilite-unread --no-init --quit-at-eof '
|
||||||
|
|
||||||
|
ENV_SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
|
||||||
|
PATH_add "$ENV_SCRIPT_DIR"
|
||||||
|
|
||||||
|
export PATH
|
||||||
|
|
||||||
|
export TF_LOG='info' # Options are: off, error, warn, info, debug, trace
|
||||||
|
export TF_LOG_PATH='_logs/terraform'
|
||||||
|
|
||||||
|
# Terraform deployment plan name
|
||||||
|
SZ_TF_DIRNAME="$(basename "$PWD")"
|
||||||
|
LOG_DIR="$SZ_TF_DIRNAME/_tf/_logs" # save for manipulation
|
||||||
|
|
||||||
|
if [[ "$SZ_TF_DIRNAME" == _tf ]]; then
|
||||||
|
SZ_TF_DIRNAME="$(basename "${PWD%/*}")"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Determine correct log location
|
||||||
|
LOG_DIR="$(echo "$LOG_DIR" | sed -Ee 's|(^_tf[^/]*/)_tf/|\1|; s|^[^/]*/||')"
|
||||||
|
[[ -d "${LOG_DIR}" ]] || mkdir "${LOG_DIR}" 2>/dev/null || true
|
||||||
|
|
||||||
|
export SZ_TF_DIRNAME SZ_TF_NAME="$SZ_TF_DIRNAME"
|
||||||
|
|
||||||
|
TFVARS_RESET_OR_BLANK=RESET
|
||||||
|
|
||||||
|
function set_tf_vars() {
|
||||||
|
if [[ "$1" == "RESET" ]]; then
|
||||||
|
unset TF_VAR_PLAN_PATH TF_VAR_OUT TF_VAR_FILE_CLI TF_CLI_ARGS
|
||||||
|
unset TF_CLI_ARGS_init TF_CLI_ARGS_validate TF_CLI_ARGS_apply
|
||||||
|
unset TF_CLI_ARGS_plan TF_CLI_ARGS_refresh TF_CLI_ARGS_destroy
|
||||||
|
else
|
||||||
|
:
|
||||||
|
fi
|
||||||
|
TF_VAR_PLAN_PATH="${TF_VAR_PLAN_PATH:-_.tmp.${SZ_TF_NAME}.tfplan}"
|
||||||
|
TF_VAR_OUT="${TF_VAR_OUT:--out ${TF_VAR_PLAN_PATH}}"
|
||||||
|
# TF_VAR_FILE_CLI="-var-file="
|
||||||
|
# TF_VAR_FILE_CLI="${TF_VAR_FILE_CLI}"
|
||||||
|
|
||||||
|
# TF_CLI_ARGS=''
|
||||||
|
# TF_CLI_ARGS_init=''
|
||||||
|
# TF_CLI_ARGS_validate=''
|
||||||
|
TF_CLI_ARGS_apply="${TF_CLI_ARGS_apply:-${TF_VAR_PLAN_PATH}}"
|
||||||
|
TF_CLI_ARGS_plan="${TF_CLI_ARGS_plan:-${TF_VAR_OUT} ${TF_VAR_FILE_CLI}}"
|
||||||
|
TF_CLI_ARGS_refresh="${TF_CLI_ARGS_refresh:-${TF_VAR_FILE_CLI}}"
|
||||||
|
# TF_CLI_ARGS_destroy=''
|
||||||
|
|
||||||
|
export TF_IN_AUTOMATION="${TF_IN_AUTOMATION:-1}"
|
||||||
|
|
||||||
|
# console
|
||||||
|
# fmt
|
||||||
|
# force-unlock
|
||||||
|
# get
|
||||||
|
# graph
|
||||||
|
# import
|
||||||
|
# login
|
||||||
|
# logout
|
||||||
|
# output
|
||||||
|
# providers
|
||||||
|
# refresh
|
||||||
|
# show
|
||||||
|
# state
|
||||||
|
# taint
|
||||||
|
# test
|
||||||
|
# untaint
|
||||||
|
# version
|
||||||
|
# workspace
|
||||||
|
export TF_VAR_PLAN_PATH TF_VAR_OUT TF_VAR_FILE_CLI TF_CLI_ARGS
|
||||||
|
export TF_CLI_ARGS_init TF_CLI_ARGS_validate TF_CLI_ARGS_apply
|
||||||
|
export TF_CLI_ARGS_plan TF_CLI_ARGS_refresh TF_CLI_ARGS_destroy
|
||||||
|
export SZ_TF_NETWORK_NAME
|
||||||
|
}
|
||||||
|
set_tf_vars "${TFVARS_RESET_OR_BLANK}"
|
||||||
|
|
||||||
|
set +e
|
|
@ -0,0 +1,86 @@
|
||||||
|
#! /usr/bin/env bash
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<USAGE
|
||||||
|
Usage:
|
||||||
|
[RESET=1] [ROOT_TF=1] prep-proj-tf [<template-name>] [<gcp-project>]
|
||||||
|
|
||||||
|
Description:
|
||||||
|
Crates gcp-project/_tf directory based on teamplate-name
|
||||||
|
When template-name is blank, only links to template-root.
|
||||||
|
When gcp-project isn't provided, the current directory will
|
||||||
|
be used if validated as a gcp-project.
|
||||||
|
|
||||||
|
ROOT_TF=1 indicates setting up the root's _tf directory, and not any
|
||||||
|
specific project.
|
||||||
|
|
||||||
|
List of templates:
|
||||||
|
USAGE
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
list-templates-for-usage
|
||||||
|
}
|
||||||
|
|
||||||
|
set -e
|
||||||
|
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
|
||||||
|
# shellcheck disable=SC1091
|
||||||
|
source "$SCRIPT_DIR/../prep-repo.sh.inc"
|
||||||
|
RUN_DIR="$PWD"
|
||||||
|
NAME="$(basename "$RUN_DIR")"
|
||||||
|
[[ "$NAME" == '_tf' ]] && RUN_DIR="$( dirname "$RUN_DIR" )"
|
||||||
|
|
||||||
|
if [[ "$1" == "--help" ]]; then
|
||||||
|
usage
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Make sure we're running on a gcp-project/project level
|
||||||
|
TST_RUN_DIR="$RUN_DIR"
|
||||||
|
[[ "$ROOT_TF" != 1 ]] || TST_RUN_DIR="${GIT_ROOT}"
|
||||||
|
[[ -z "$2" ]] || TST_RUN_DIR="${GIT_ROOT}/$2"
|
||||||
|
PROJ_NAME="$(basename "${TST_RUN_DIR}")"
|
||||||
|
if [[ "$TST_RUN_DIR" == "$GIT_ROOT" ]]; then
|
||||||
|
if [[ "$ROOT_TF" == 1 ]]; then
|
||||||
|
printf "Setting up git root _tf\n"
|
||||||
|
else
|
||||||
|
printf '%s\n' \
|
||||||
|
'Preparing git root failed.' \
|
||||||
|
'Are you in the correct location?' \
|
||||||
|
'Did you forget to pass ROOT_TF=1?'
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
if [[ "$TST_RUN_DIR" != "${GIT_ROOT}/${PROJ_NAME}" ]]; then
|
||||||
|
printf "%s is not a valid project path\n" "$TST_RUN_DIR"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
RUN_DIR="${GIT_ROOT}/${PROJ_NAME}"
|
||||||
|
|
||||||
|
TMP=$("$SCRIPT_DIR/../prep-repo" "$PROJ_NAME")
|
||||||
|
echo "$TMP"
|
||||||
|
echo "$TMP" | grep -q "Skipped" && exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "Preparing ${PROJ_NAME}"
|
||||||
|
mkdir -p "${RUN_DIR}/_tf/_logs"
|
||||||
|
RUN_DIR="${RUN_DIR}/_tf"
|
||||||
|
|
||||||
|
remove_links_from_run_dir_on_RESET
|
||||||
|
|
||||||
|
[[ "$ROOT_TF" == "1" ]] \
|
||||||
|
|| safe_link "direnv/envrc.project-tf" "$RUN_DIR/.envrc"
|
||||||
|
|
||||||
|
TMPL_NAME="${1}"
|
||||||
|
SAFE_ROOT="$SZ_COMMON_PATH/tf/templates" link_templates
|
||||||
|
SAFE_ROOT="$SZ_COMMON_PATH/../tf/templates" link_templates
|
||||||
|
|
||||||
|
if [[ -n "${TMPL_NAME}" && $LINKED_TMPL = 0 ]]; then
|
||||||
|
printf "Failed to initialize %s template, %s\n" \
|
||||||
|
"${TMPL_NAME}" \
|
||||||
|
"as the path does not exist"
|
||||||
|
fi
|
||||||
|
|
||||||
|
[[ -r "$RUN_DIR/README.md" ]] || touch "$RUN_DIR/README.md"
|
||||||
|
[[ -r "$RUN_DIR/DEPLOYMENT-STEPS.md" ]] || touch "$RUN_DIR/DEPLOYMENT-STEPS.md"
|
|
@ -0,0 +1,86 @@
|
||||||
|
#! /usr/bin/env bash
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<USAGE
|
||||||
|
Usage:
|
||||||
|
[RESET=1] prep-tf-host <template-name> <gcp-project/tf-name>
|
||||||
|
|
||||||
|
Description:
|
||||||
|
Crates gcp-project/tf-name directory based on teamplate-name
|
||||||
|
When template-name is blank, only links to template-root.
|
||||||
|
When gcp-project/tf-name will grab current directory name and if
|
||||||
|
it is a valid location, it will use the location to feed the
|
||||||
|
script's logic.
|
||||||
|
|
||||||
|
List of templates:
|
||||||
|
USAGE
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
list-templates-for-usage
|
||||||
|
}
|
||||||
|
|
||||||
|
set -e
|
||||||
|
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
|
||||||
|
# shellcheck disable=SC1091
|
||||||
|
source "$SCRIPT_DIR/../prep-repo.sh.inc"
|
||||||
|
RUN_DIR="$PWD"
|
||||||
|
NAME="$(basename "$RUN_DIR")"
|
||||||
|
[[ "$NAME" == '_tf' ]] && RUN_DIR="$( dirname "$RUN_DIR" )"
|
||||||
|
|
||||||
|
if [[ "$1" == "--help" ]]; then
|
||||||
|
usage
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Make sure we're running on a gcp-project/project level
|
||||||
|
TST_RUN_DIR="$RUN_DIR"
|
||||||
|
|
||||||
|
[[ -z "$2" ]] || TST_RUN_DIR="${GIT_ROOT}/$2"
|
||||||
|
PATH_ARG="$(basename "$(dirname "${TST_RUN_DIR}")")/$(basename "${TST_RUN_DIR}")"
|
||||||
|
if [[ "$TST_RUN_DIR" != "${GIT_ROOT}/${PATH_ARG}" ]]; then
|
||||||
|
printf "%s is not a valid path\n" "$TST_RUN_DIR"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
PROJ_NAME="$(basename "$(dirname "${PATH_ARG}")")"
|
||||||
|
TF_NAME="$(basename "${PATH_ARG}")"
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
if [[ "$PROJ_NAME/$TF_NAME" != "${PATH_ARG}" ]]; then
|
||||||
|
printf "ERROR %s does not match argument %s, %s\n" \
|
||||||
|
"$PROJ_NAME/$TF_NAME" "${PATH_ARG}" \
|
||||||
|
"this does not seem to be a valid project path."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
RUN_DIR="${GIT_ROOT}/${PATH_ARG}"
|
||||||
|
|
||||||
|
TMP=$("$SCRIPT_DIR/../prep-repo" "$PROJ_NAME")
|
||||||
|
echo "$TMP"
|
||||||
|
echo "$TMP" | grep -q "Skipped" && exit 1
|
||||||
|
|
||||||
|
|
||||||
|
echo "Preparing ${PATH_ARG}"
|
||||||
|
mkdir -p "${RUN_DIR}/_tf/_logs"
|
||||||
|
|
||||||
|
|
||||||
|
remove_links_from_run_dir_on_RESET
|
||||||
|
|
||||||
|
|
||||||
|
safe_link "direnv/envrc.project-tf" "$RUN_DIR/.envrc"
|
||||||
|
|
||||||
|
TMPL_NAME="${1}"
|
||||||
|
RUN_DIR="$RUN_DIR/_tf" SAFE_ROOT="$SZ_COMMON_PATH/tf/templates" link_templates
|
||||||
|
RUN_DIR="$RUN_DIR/_tf" SAFE_ROOT="$SZ_COMMON_PATH/../tf/templates" link_templates
|
||||||
|
|
||||||
|
if [[ -n "${TMPL_NAME}" && $LINKED_TMPL = 0 ]]; then
|
||||||
|
printf "Failed to initialize %s template, %s\n" \
|
||||||
|
"${TMPL_NAME}" \
|
||||||
|
"as the path does not exist"
|
||||||
|
fi
|
||||||
|
|
||||||
|
[[ -r "$RUN_DIR/README.md" ]] || touch "$RUN_DIR/README.md"
|
||||||
|
[[ -r "$RUN_DIR/DEPLOYMENT-STEPS.md" ]] || touch "$RUN_DIR/DEPLOYMENT-STEPS.md"
|
|
@ -0,0 +1,33 @@
|
||||||
|
#! /usr/bin/env bash
|
||||||
|
set -e
|
||||||
|
|
||||||
|
HOST=$(hostname)
|
||||||
|
CERT_PREFIX="certs/whoami-$HOST"
|
||||||
|
CERT_KEY="${CERT_PREFIX}.key"
|
||||||
|
CERT_CRT="${CERT_PREFIX}.crt"
|
||||||
|
|
||||||
|
mkdir -p certs
|
||||||
|
|
||||||
|
if [[ ! -r "$CERT_CRT" || ! -r "$CERT_KEY" ]]; then
|
||||||
|
|
||||||
|
openssl genrsa -des3 -out "$CERT_KEY" \
|
||||||
|
-passout "pass:keypassphrase" \
|
||||||
|
4096
|
||||||
|
|
||||||
|
openssl req -x509 \
|
||||||
|
-key "$CERT_KEY" -passin "pass:keypassphrase" \
|
||||||
|
-sha256 -days 365 -subj "/C=$HOSTWhoAmI/ST=NC/L=Morrisville/O=acme/OU=devops/CN=whoami.devops" \
|
||||||
|
-out "$CERT_CRT" \
|
||||||
|
-passout "pass:pempassphrase"
|
||||||
|
|
||||||
|
chmod 664 "$CERT_CRT"
|
||||||
|
fi
|
||||||
|
|
||||||
|
#podman pull 'ghcr.io/traefik/whoami:v1.10.1'
|
||||||
|
podman pull 'ghcr.io/traefik/whoami:latest'
|
||||||
|
docker run --detach --restart unless-stopped \
|
||||||
|
--name whoami-08443 \
|
||||||
|
-p 8443:443 \
|
||||||
|
-v "$PWD/certs:/certs" \
|
||||||
|
traefik/whoami -cert "/$CERT_CRT" -key "/$CERT_KEY"
|
||||||
|
|
|
@ -0,0 +1,13 @@
|
||||||
|
#! /usr/bin/env bash
|
||||||
|
|
||||||
|
function _switch-dbg-tf-on() {
|
||||||
|
local OFF_TF ON_TF
|
||||||
|
for OFF_TF in dbg.*.tf.off; do
|
||||||
|
ON_TF="_.${OFF_TF%\.off}"
|
||||||
|
[[ ! -r "$ON_TF" ]] && ln -s "$OFF_TF" "$ON_TF"
|
||||||
|
echo "$ON_TF switched on."
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
_switch-dbg-tf-on "${@}"
|
||||||
|
unset _switch-dbg-tf-on
|
|
@ -0,0 +1,32 @@
|
||||||
|
#! /usr/bin/env bash
|
||||||
|
# shellcheck disable=SC1091
|
||||||
|
eval "$(. _tf_aux_functions)"
|
||||||
|
|
||||||
|
function _tf() {
|
||||||
|
[[ -z "$TF_LOG_TS" ]] && _tfSetLogTS
|
||||||
|
local NAME=$1
|
||||||
|
[[ "${*}" =~ "-destroy" ]] && NAME="$1-destroy"
|
||||||
|
|
||||||
|
echo "===_logs/0_$NAME.log===" > _logs/0_0_lastrun.log
|
||||||
|
echo "===_logs/${TF_LOG_TS}_$NAME.log===" \
|
||||||
|
| tee --append _logs/0_0_lastrun.log \
|
||||||
|
> "_logs/0_$NAME.log"
|
||||||
|
[[ -z "$SZ_DEBUG" ]] || echo "Executing: terraform ${*}"
|
||||||
|
{
|
||||||
|
{ \
|
||||||
|
terraform "${@}" 2>&1 || _tf_save_exitCode $?
|
||||||
|
} | tee "_logs/${TF_LOG_TS}_$NAME.log" \
|
||||||
|
| awk 'BEGIN {p=1}; /<<\W*EOT/ {print; p=0}; /^\W*EOT/ {p=1}; p; fflush();' \
|
||||||
|
| tee --append _logs/0_0_lastrun.log \
|
||||||
|
>> "_logs/0_$NAME.log"
|
||||||
|
|
||||||
|
echo "===FULLSTOP===" >> _logs/0_0_lastrun.log
|
||||||
|
} &
|
||||||
|
|
||||||
|
less-tf -
|
||||||
|
}
|
||||||
|
|
||||||
|
_tf "${@}"
|
||||||
|
unset _tf
|
||||||
|
|
||||||
|
eval "$( _tf_exit_code )"
|
|
@ -0,0 +1,26 @@
|
||||||
|
#! /usr/bin/env bash
|
||||||
|
|
||||||
|
function _tf-pre-plan() {
|
||||||
|
[ -z "$SZ_TF_NAME" ] \
|
||||||
|
&& echo "ERROR: SZ_TF_NAME isn't declared!" \
|
||||||
|
&& return 1
|
||||||
|
|
||||||
|
[[ "${*}" =~ .*--no-delete.* ]] || find . -name "_.gen.*" -delete
|
||||||
|
|
||||||
|
for TF_TMPL_FILE in $(bash -c 'shopt -s nullglob; echo *.sz_tmpl'); do
|
||||||
|
local OUT_FILE="${TF_TMPL_FILE%\.sz_tmpl}"
|
||||||
|
OUT_FILE="_.gen.${SZ_TF_NAME}.${OUT_FILE#_sz\.}"
|
||||||
|
|
||||||
|
echo "Generating ${OUT_FILE} file..."
|
||||||
|
printf "%s\n" \
|
||||||
|
"# DO NOT EDIT THIS FILE!!!! " \
|
||||||
|
"# This file was autogenerated by \`_tf-pre-plan\` from ${TF_TMPL_FILE}" \
|
||||||
|
"" \
|
||||||
|
"$( envsubst < "${TF_TMPL_FILE}" )" \
|
||||||
|
> "${OUT_FILE}"
|
||||||
|
done
|
||||||
|
printf "\n\n" >&2
|
||||||
|
}
|
||||||
|
|
||||||
|
_tf-pre-plan "${@}"
|
||||||
|
unset _tf-pre-plan
|
|
@ -0,0 +1,12 @@
|
||||||
|
#! /usr/bin/env bash
|
||||||
|
# shellcheck disable=SC1091
|
||||||
|
eval "$(. _tf_aux_functions)"
|
||||||
|
|
||||||
|
. tf apply "${@}" && _tfSetLogTS
|
||||||
|
eval "$( _tf_exit_code )"
|
||||||
|
. tf-extract \
|
||||||
|
| tee "_logs/${TF_LOG_TS}_state_ids.json" \
|
||||||
|
> _logs/0_9_last_state_ids.json \
|
||||||
|
&& echo "_logs/${TF_LOG_TS}_state_ids.json written" >&2
|
||||||
|
|
||||||
|
eval "$( _tf_exit_code )"
|
|
@ -0,0 +1,38 @@
|
||||||
|
#! /usr/bin/env bash
|
||||||
|
# shellcheck disable=SC1091
|
||||||
|
eval "$(. _tf_aux_functions)"
|
||||||
|
|
||||||
|
final_render() {
|
||||||
|
if [ "$1" = "--clip" ]; then
|
||||||
|
jq -r '"- \(.name):|\(.id)"' \
|
||||||
|
| column -ts'|' | cut -c -${2:-${COLUMNS:-$(tput cols)}}
|
||||||
|
else
|
||||||
|
jq
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
terraform show -json | jq \
|
||||||
|
| tee "_logs/${TF_LOG_TS}_state.json" \
|
||||||
|
| tee "_logs/0_state.json" \
|
||||||
|
| jq '
|
||||||
|
[[.values.root_module.resources,
|
||||||
|
(.values.root_module.child_modules // [] | .[].resources // [])
|
||||||
|
] | map(.[] ) | .[]
|
||||||
|
| select( .mode != "data" )
|
||||||
|
| {
|
||||||
|
name: .address,
|
||||||
|
id: (
|
||||||
|
if( .type == "google_storage_bucket" ) then
|
||||||
|
"\(.values.project)/\(.values.id)"
|
||||||
|
else
|
||||||
|
.values.id
|
||||||
|
end
|
||||||
|
)
|
||||||
|
}]
|
||||||
|
' \
|
||||||
|
| tee "_logs/${TF_LOG_TS}_state_ids.json" \
|
||||||
|
| tee "_logs/0_9_last_state_ids.json" \
|
||||||
|
| final_render "$@"
|
||||||
|
|
||||||
|
eval "$( _tf_exit_code )"
|
||||||
|
|
|
@ -0,0 +1,13 @@
|
||||||
|
#! /usr/bin/env bash
|
||||||
|
# shellcheck disable=SC1091
|
||||||
|
eval "$(. _tf_aux_functions)"
|
||||||
|
|
||||||
|
IDs_JSON="${1:--}"
|
||||||
|
|
||||||
|
jq -r ' .[] |
|
||||||
|
"tf state rm \"\(.name)\";\n tf import \"\(.name)\"% \"\(.id)\";"
|
||||||
|
' "$IDs_JSON" \
|
||||||
|
| awk '{printf "%s%s\n", (NR==1 ? "#! /usr/bin/env bash\n\n" : ""), $0;}' \
|
||||||
|
| column -ts'%'
|
||||||
|
|
||||||
|
eval "$( _tf_exit_code )"
|
|
@ -0,0 +1,13 @@
|
||||||
|
#! /usr/bin/env bash
|
||||||
|
eval "$(. _tf_aux_functions)"
|
||||||
|
|
||||||
|
function _tf-init() {
|
||||||
|
tf-_pre-plan "${@}"
|
||||||
|
|
||||||
|
tf0 init "${@}"
|
||||||
|
}
|
||||||
|
|
||||||
|
_tf-init "${@}"
|
||||||
|
unset _tf-init
|
||||||
|
|
||||||
|
eval "$( _tf_exit_code )"
|
|
@ -0,0 +1,14 @@
|
||||||
|
#! /usr/bin/env bash
|
||||||
|
resources=$()
|
||||||
|
|
||||||
|
# Loop over resources and output name and ID pairs
|
||||||
|
terraform state list | grep -Ev '^data\.' | while read -r r; do
|
||||||
|
printf 'tf import %s %s\n' \
|
||||||
|
"$r" \
|
||||||
|
"$( terraform state show "$r" \
|
||||||
|
| sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]//g" \
|
||||||
|
| grep -E "^\W*(id|self-link)\W*=" \
|
||||||
|
| head -1 \
|
||||||
|
| awk '{print $3}' \
|
||||||
|
)"
|
||||||
|
done
|
|
@ -0,0 +1,31 @@
|
||||||
|
#! /usr/bin/env bash
|
||||||
|
# shellcheck disable=SC1091
|
||||||
|
eval "$(. _tf_aux_functions)"
|
||||||
|
|
||||||
|
eval "$( _tf_exit_code )"
|
||||||
|
_tf-plan() {
|
||||||
|
[[ -z "$SZ_DEBUG" ]] || echo "DEBUG: tf-plan ${*}"
|
||||||
|
local _TF_TARGET=""
|
||||||
|
# shellcheck disable=SC2153,SC2086 # TF_TARGET references an external env
|
||||||
|
[[ ${#TF_TARGET} -gt 0 ]] && _TF_TARGET="$(printf -- '--target=%s ' ${TF_TARGET})"
|
||||||
|
# shellcheck disable=SC2086 # word splitting is desired here
|
||||||
|
[[ "${*}" =~ "-destroy" ]] && [[ ${#TF_DESTROY_TARGET} -gt 0 ]] && _TF_TARGET="$(printf -- '--target=%s ' ${TF_DESTROY_TARGET})"
|
||||||
|
|
||||||
|
tf-_pre-plan "${@}"
|
||||||
|
# shellcheck disable=SC2086 # word splitting is desired here
|
||||||
|
tf plan ${_TF_TARGET} "${@}"
|
||||||
|
}
|
||||||
|
|
||||||
|
[[ -z "$TF_LOG_TS" ]] && _tfSetLogTS
|
||||||
|
_tf-plan "${@}"
|
||||||
|
unset _tf-plan
|
||||||
|
|
||||||
|
LOG_NAME="_logs/${TF_LOG_TS}_plan"
|
||||||
|
[[ "${*}" =~ "-destroy" ]] && LOG_NAME="${LOG_NAME}-destroy"
|
||||||
|
LOG_NAME="${LOG_NAME}.log"
|
||||||
|
|
||||||
|
[[ ! -r /tmp/TF_EXITCODE ]] \
|
||||||
|
&& grep -E '^(.\[1m)? # .* (forces|(must|will) be)' "${LOG_NAME}" \
|
||||||
|
| tee --append _logs/0_0_lastrun.log
|
||||||
|
|
||||||
|
eval "$( _tf_exit_code )"
|
|
@ -0,0 +1,6 @@
|
||||||
|
#! /usr/bin/env bash
|
||||||
|
eval "$(. _tf_aux_functions)"
|
||||||
|
|
||||||
|
. tf0-plan --destroy "${@}"
|
||||||
|
|
||||||
|
eval "$( _tf_exit_code )"
|
|
@ -0,0 +1,13 @@
|
||||||
|
#! /usr/bin/env bash
|
||||||
|
eval "$(. _tf_aux_functions)"
|
||||||
|
|
||||||
|
function _tf0() {
|
||||||
|
_tfSetLogTS
|
||||||
|
[ $# -eq 0 ] && return
|
||||||
|
tf "$@"
|
||||||
|
}
|
||||||
|
|
||||||
|
_tf0 "${@}"
|
||||||
|
unset _tf0
|
||||||
|
|
||||||
|
eval "$( _tf_exit_code )"
|
|
@ -0,0 +1,7 @@
|
||||||
|
#! /usr/bin/env bash
|
||||||
|
eval "$(. _tf_aux_functions)"
|
||||||
|
|
||||||
|
_tfSetLogTS
|
||||||
|
. tf-plan "${@}"
|
||||||
|
|
||||||
|
eval "$( _tf_exit_code )"
|
|
@ -0,0 +1,2 @@
|
||||||
|
#! /usr/bin/env bash
|
||||||
|
. tf-apply "${@}"
|
|
@ -0,0 +1,14 @@
|
||||||
|
#! /usr/bin/env bash
|
||||||
|
|
||||||
|
function _tfcontext() {
|
||||||
|
[[ -n $(find $PWD -name '*.tf') ]] || return -1
|
||||||
|
[[ -d _logs ]] || mkdir _logs
|
||||||
|
TF_VAR_FILE_NAME=${TF_VAR_FILE_CLI-:$(basename $PWD).tfvars}
|
||||||
|
[[ -r $TF_VAR_FILE_NAME ]] || unset TF_VAR_FILE_NAME
|
||||||
|
TF_VAR_FILE_CLI=${TF_VAR_FILE_CLI-:-var-file='$TF_VAR_FILE_NAME'}
|
||||||
|
|
||||||
|
basename $PWD
|
||||||
|
}
|
||||||
|
|
||||||
|
_tfcontext "${@}"
|
||||||
|
unset _tfcontext
|
|
@ -0,0 +1,2 @@
|
||||||
|
#! /usr/bin/env bash
|
||||||
|
. tf-plan "${@}"
|
|
@ -0,0 +1,2 @@
|
||||||
|
#! /usr/bin/env bash
|
||||||
|
. tf-plan-destroy "${@}"
|
Loading…
Reference in New Issue