This the local development environment I run on my home server, right now I'm using it for working with OLLAMA AI, but it's just a way to organize docker-compose to make it easy to add new services as needed for devops type of setups (CICD, git/db/web servers, docker and pip registries, TIG stack monitoring).
You can re-run make fairly safely to push updates and changes since it uses external volumes that survive a docker-compose down, but if you are keeping anything important to you in any of the services, I would back them up somewhere just in-case.
- Linux
- Ansible
- Docker
- InfluxDB Cli
- Python3
Note: I'm running this on a 12core Debian machine with 32G ram and ZFS docker volumes, docker eats a lot of memory running the monitoring stack, so comment it out if the overhead is too much, the influxdb and telegraf settings might need to be tweaked for better performance and feel free to PR any improvements you find.
Replace <YOUR_SERVER_IP> with the ip of your docker server
SERVER_IP=<YOUR_SERVER_IP> cat >.env<<EOF DEVSTACK_DIR=/opt/devstack DRONE_DATABASE_DRIVER=postgres DRONE_DATABASE_SOURCE=postgres://drone:drone@postgres:5432/drone?sslmode=disable DRONE_GOGS_SERVER=http://gogs:3000 DRONE_RPC_SECRET=derpy-derp-derp GOGS_ADMIN=gadmin GRAFANA_ADMIN_PASS=adm1n GRAFANA_ADMIN_USER=admin INFLUX_DB_BUCKET=telegraf INFLUX_DB_ORG=null INFLUX_DB_TOKEN=<Created Token> INFLUX_DB_PASSWORD=influxdb INFLUX_DB_USER=admin INFLUX_URL=http://${SERVER_IP}:8086 INFLUXDB_DB=telegraf INFLUXDB_PASSWORD=influxdb INFLUXDB_USER=admin NGINX_HOST=${SERVER_IP} [email protected] PGADMIN_PASS=adm1n EOF cat >.vars.yml<<EOF gogs_domain: ${SERVER_IP} gogs_pass: gadm1n gogs_root_url: http://${SERVER_IP}:3000 gogs_secret_key: derp-derpy-derp gogs_url: 127.0.0.1:3000 gogs_user: gadmin EOF The DB password for gogs and drone is set via the initdb sql files, change them there if changing the vars above.
- Build the images and run containers:
make docker-compose ps Name Command State Ports ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ drone_runner /bin/drone-runner-docker Up 0.0.0.0:8300->3000/tcp,:::8300->3000/tcp drone_server /bin/drone-server Up 0.0.0.0:8443->443/tcp,:::8443->443/tcp, 0.0.0.0:8000->80/tcp,:::8000->80/tcp gogs /app/gogs/docker/start.sh ... Up (healthy) 0.0.0.0:3022->22/tcp,:::3022->22/tcp, 0.0.0.0:3000->3000/tcp,:::3000->3000/tcp grafana /app/entrypoint.sh Up 3000/tcp, 0.0.0.0:3080->3080/tcp,:::3080->3080/tcp influxdb /app/entrypoint.sh Up 0.0.0.0:8086->8086/tcp,:::8086->8086/tcp mongo docker-entrypoint.sh mongod Up 0.0.0.0:27017->27017/tcp,:::27017->27017/tcp mongo-ui /sbin/tini -- /docker-entr ... Up 0.0.0.0:8081->8081/tcp,:::8081->8081/tcp ollama_server /bin/ollama serve Up 0.0.0.0:11434->11434/tcp,:::11434->11434/tcp ollama_webui bash start.sh Up (healthy) 0.0.0.0:8080->8080/tcp,:::8080->8080/tcp pgadmin /entrypoint.sh Up 443/tcp, 0.0.0.0:5050->80/tcp,:::5050->80/tcp postgres /entrypoint.sh postgres Up 0.0.0.0:5432->5432/tcp,:::5432->5432/tcp pypi_server /entrypoint.sh Up 0.0.0.0:9000->8080/tcp,:::9000->8080/tcp registry /entrypoint.sh /etc/docker ... Up 0.0.0.0:5000->5000/tcp,:::5000->5000/tcp registry-ui /docker-entrypoint.sh ngin ... Up 0.0.0.0:5080->80/tcp,:::5080->80/tcp selenium_server /opt/bin/entry_point.sh Up 0.0.0.0:4444->4444/tcp,:::4444->4444/tcp, 5900/tcp, 0.0.0.0:7900->7900/tcp,:::7900->7900/tcp, 9000/tcp telegraf /entrypoint.sh telegraf Up 8092/udp, 8094/tcp, 8125/udp unit /usr/local/bin/docker-entr ... Up 0.0.0.0:80->80/tcp,:::80->80/tcp, 0.0.0.0:9080->9080/tcp,:::9080->9080/tcp
- Just run make down
make down
Create an api token
docker exec -it influxdb influx auth create --org null --all-access -d apitokenUpdate .env file INFLUX_DB_TOKEN value
Rebuild grafana and telegraf
./rebuild_service.sh grafana ./rebuild_service.sh telegraf
Run app target
make appConfigure the unit server
make configif there's an error, you can check the container logs:
docker logs unit -for exec into the container and look around:
docker exec -ti -w /www unit /bin/bash root@unit:/www# su - unit unit@unit:~$Restart Unit Apps (as needed) The unit apps will need to be restarted if using a mount and the code is changed on disk.
# fastapi $ make app_restart_fapi
I recommend using pypi-mirror to populate the pypi server with any requirements you might have, this will save you some downloading when recreating containers.
I create a container with the following mounts:
/opt/devstack/packages:/data/packages /opt/pypi_mirror/pip_dir:/root/.cache/pip /opt/pypi_mirror/tmp_dir:/tmp This mounts the packages dir used by the pypi_server
Also copy over a requirements.txt with a list of pip packages you want to mirror, pypy-miror will pull the dependancies, so you don't need to be overly explict about stuff unless you want particular versions.
Now exec into the container (or setup an entrypoint script) and run the pypi-mirror like this:
$ cd /data $ pypi-mirror download -d packages -r requirements.txt $ pypi-mirror create -d packages -m simple Now whereever you need to install pip packages use a pip.conf like this:
[global] index-url=http://172.16.16.1:9000/simple trusted-host=172.16.16.1 This is the internal gateway IP of the docker server, it should work fine for anything in the devstack network.
Just note that any updates to the packages on disk (mirroring) requires the pypi-server to be restarted.
$ export SERVER_IP=X.X.X.X $ cat <<EOC # Unit Server (static) http://${SERVER_IP}/ # Gogs UI http://${SERVER_IP}:3000/ # Selenium Server http://${SERVER_IP}:4444/ # pgAdmin - Postgres admin ui http://${SERVER_IP}:5050/ # Docker Registry http://${SERVER_IP}:5080/ # Drone Server - CICD, you need to enable and trust repos for them to build http://${SERVER_IP}:8000/ # Ollama WebUI http://${SERVER_IP}:8080/ # MongoDB Express UI (default login admin:pass) http://${SERVER_IP}:8081/ # Influxdb - has dashboards you can use to see monitoring datta http://${SERVER_IP}:8086/ # PIP Packages http://${SERVER_IP}:9000/packages/ # Unit Server (fapi) http://${SERVER_IP}:9080/ EOC