Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .buildpacks

This file was deleted.

38 changes: 38 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# VCS and CI
.git
.gitignore
.github

# Dependencies and build output (rebuilt inside the image)
node_modules/
oh_queue/static/public/
oh_queue/static/.webassets-cache/

# Local Python environments and caches
venv/
.venv
__pycache__/
*.py[cod]
.pytest_cache/

# Secrets and local config - never bake these into an image
.env*
*credentials*.json

# Local databases and exports
*.db
*.sql

# Tests and dev-only files
tests/
pytest.ini
.coverage
coverage.xml
requirements-dev.txt
codecov.yml
.flake8
.flowconfig
mise.toml
docs/
README.md
**/.DS_Store
9 changes: 7 additions & 2 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,17 @@
# dev or prod. Required: the app won't start without it.
ENV=dev
FLASK_APP=manage.py
EVENTLET_NO_GREENDNS=yes

# Required outside development.
SECRET_KEY=
# Required outside development; in development the app uses app.db (SQLite).
# Required outside development, unless CLOUD_SQL_INSTANCE is set; in
# development the app uses app.db (SQLite).
DATABASE_URL=
# Deployments connect to Cloud SQL as the service account they run as (IAM
# database auth, so no password) instead of using DATABASE_URL.
# CLOUD_SQL_INSTANCE=project:region:instance
# DB_USER=service-account@project.iam
# DB_NAME=oh

# Canvas developer key. Locally, use the UC Berkeley sandbox key:
# https://ucberkeleysandbox.instructure.com/
Expand Down
127 changes: 127 additions & 0 deletions .github/workflows/pr-staging.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
name: PR Preview Deployment

on:
pull_request:
types: [opened, synchronize, reopened, closed]

permissions:
contents: read

env:
# Keyless auth via Workload Identity Federation; only this repository may
# impersonate the deployer service account.
WORKLOAD_IDENTITY_PROVIDER: projects/137456085712/locations/global/workloadIdentityPools/github/providers/berkeley-cdss
DEPLOYER_SERVICE_ACCOUNT: oh-gh-actions-deployer@staging-cdss-web-apps.iam.gserviceaccount.com

# One deploy per PR at a time; a newer push supersedes an in-progress deploy.
concurrency:
group: preview-oh-pr-${{ github.event.number }}
cancel-in-progress: true

# Fork PRs get no OIDC token, and their code shouldn't run with staging secrets,
# so previews only deploy for branches in this repository. Dependabot PRs are
# skipped too: they run their own changed workflows before review, so a bumped
# action pin would otherwise get the deployer's GCP access unreviewed.
jobs:
cleanup-preview-service:
if: github.event.action == 'closed' && github.event.pull_request.head.repo.full_name == github.repository && github.event.pull_request.user.login != 'dependabot[bot]'
runs-on: ubuntu-latest
permissions:
# Lets google-github-actions/auth exchange GitHub's OIDC token for GCP credentials.
id-token: write
steps:
- id: auth
uses: google-github-actions/auth@7c6bc770dae815cd3e89ee6cdf493a5fab2cc093 # v3.0.0
with:
workload_identity_provider: ${{ env.WORKLOAD_IDENTITY_PROVIDER }}
service_account: ${{ env.DEPLOYER_SERVICE_ACCOUNT }}

- name: Delete Cloud Run Preview Service
run: |
gcloud run services delete preview-oh-pr-${{ github.event.number }} \
--region=us-west1 \
--quiet || true

deploy-preview-service:
if: github.event.action != 'closed' && github.event.pull_request.head.repo.full_name == github.repository && github.event.pull_request.user.login != 'dependabot[bot]'
runs-on: ubuntu-latest
permissions:
contents: read
# Lets google-github-actions/auth exchange GitHub's OIDC token for GCP credentials.
id-token: write
# Lets the deploy job comment the preview URL on the PR.
issues: write
pull-requests: write
steps:
- name: Checkout Code
uses: actions/checkout@v7
with:
ref: ${{ github.head_ref }}

- id: auth
uses: google-github-actions/auth@7c6bc770dae815cd3e89ee6cdf493a5fab2cc093 # v3.0.0
with:
workload_identity_provider: ${{ env.WORKLOAD_IDENTITY_PROVIDER }}
service_account: ${{ env.DEPLOYER_SERVICE_ACCOUNT }}

- name: Set up Cloud SDK
uses: google-github-actions/setup-gcloud@aa5489c8933f4cc7a4f7d45035b3b1440c9c10db # v3.0.1

- name: Deploy to Cloud Run (Preview)
id: deploy
uses: google-github-actions/deploy-cloudrun@2028e2d7d30a78c6910e0632e48dd561b064884d # v3.0.1
with:
service: preview-oh-pr-${{ github.event.number }}
region: us-west1
source: .
# The image is built as oh-build, which can only build, not as the default compute
# service account, which has Editor on the project.
flags: '--allow-unauthenticated --service-account=oh-app@staging-cdss-web-apps.iam.gserviceaccount.com --build-service-account=projects/staging-cdss-web-apps/serviceAccounts/oh-build@staging-cdss-web-apps.iam.gserviceaccount.com --max-instances=3'
# RUN_MIGRATIONS is never set here: previews share the staging database.
env_vars: |
ENV=prod
# Lets the app explain that a PR adding a migration can't run as a preview.
PREVIEW_PR=${{ github.event.number }}
CANVAS_SERVER_URL=https://ucberkeley.test.instructure.com/
ADMIN_OVERRIDE_CANVAS_COURSE_ID=1549197
# Cloud SQL IAM database auth as the service's own account; no password.
CLOUD_SQL_INSTANCE=staging-cdss-web-apps:us-west1:staging-cdss-web-apps-2
DB_USER=oh-app@staging-cdss-web-apps.iam
DB_NAME=oh
secrets: |
CANVAS_CLIENT_ID=oh-canvas-client-id:latest
CANVAS_CLIENT_SECRET=oh-canvas-client-secret:latest
SECRET_KEY=oh-secret-key:latest
LLM_TIPS_ENDPOINT=oh-llm-tips-endpoint:latest
LLM_TIPS_AUTH_TOKEN=oh-llm-tips-auth-token:latest

- name: Allow Unauthenticated Access (allUsers)
run: |
gcloud run services add-iam-policy-binding preview-oh-pr-${{ github.event.number }} \
--region=us-west1 \
--member="allUsers" \
--role="roles/run.invoker"

- name: Comment PR with Preview URL
uses: actions/github-script@v7
env:
PREVIEW_URL: https://preview-oh-pr-${{ github.event.number }}-137456085712.us-west1.run.app
with:
script: |
// The preview URL is stable for the life of the PR, so only comment once.
const marker = '<!-- oh-pr-preview -->';
const comments = await github.paginate(github.rest.issues.listComments, {
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
});
if (comments.some(c => c.body?.includes(marker))) {
core.info('Preview URL already commented; skipping.');
return;
}
await github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: `${marker}\n🚀 **Staging Preview Deployed!**\n\nURL: ${process.env.PREVIEW_URL}`
});
86 changes: 74 additions & 12 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,19 +1,81 @@
FROM python:3.14-slim
# syntax=docker/dockerfile:1

# Image for deploying the office hours queue to Google Cloud Run, like seating's
# and sections'.

# ---------------------------------------------------------------------------
# Node packages
#
# Babel, which compiles the React components into one script in the build stage.
# ---------------------------------------------------------------------------
FROM node:24-slim AS node

WORKDIR /build
COPY package.json yarn.lock ./
# Babel is in dependencies; devDependencies only has Flow, which the build doesn't use.
RUN yarn install --frozen-lockfile --production

ENV APP_HOME /app
ENV FLASK_APP=manage.py
ENV PYTHONPATH=$PYTHONPATH:.
# ---------------------------------------------------------------------------
# Build stage
#
# Installs the Python dependencies into a venv that is copied into the runtime
# stage, so pip and its build cache stay out of the shipped image, then builds
# the frontend: in production the app serves oh_queue/static/public/common.js
# and style.css as they are (oh_queue/assets.py), so `flask build` makes them
# here, with Node, which the runtime image doesn't need.
# ---------------------------------------------------------------------------
FROM python:3.14-slim AS builder

WORKDIR $APP_HOME
COPY . ./
ENV PIP_DISABLE_PIP_VERSION_CHECK=1 \
PIP_NO_CACHE_DIR=1

RUN python -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"

# Copied on its own so the dependency layer is only rebuilt when the pins change.
COPY requirements.txt ./
RUN pip install -r requirements.txt

# execution permissions
RUN chmod +x /app/entrypoint.sh
COPY --from=node /usr/local/bin/node /usr/local/bin/node
WORKDIR /build
COPY --from=node /build/node_modules ./node_modules
COPY . .
# `flask build` refuses to run in production, and in development needs no
# settings: it renders the queue page once, which compiles the bundles.
RUN ENV=dev FLASK_APP=manage.py PYTHONPATH=. flask build

# ---------------------------------------------------------------------------
# Runtime stage
# ---------------------------------------------------------------------------
FROM python:3.14-slim

RUN useradd --create-home --uid 1000 app

COPY --from=builder /opt/venv /opt/venv

ENV PATH="/opt/venv/bin:$PATH" \
PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
PYTHONPATH=. \
FLASK_APP=manage.py \
PORT=8080

WORKDIR /app
# Owned by the app user: webassets writes its cache in oh_queue/static.
COPY --chown=app:app . .
COPY --from=builder --chown=app:app /build/oh_queue/static/public ./oh_queue/static/public

USER app

EXPOSE 8080

# Set the entrypoint
ENTRYPOINT ["/app/entrypoint.sh"]
HEALTHCHECK --interval=30s --timeout=5s --start-period=20s --retries=3 \
CMD python -c 'import os, urllib.request; urllib.request.urlopen("http://127.0.0.1:" + os.environ.get("PORT", "8080") + "/health")'

# The page polls the server, so plain threads are enough; eventlet isn't needed.
CMD gunicorn -b :$PORT --workers 4 --threads 8 --timeout 120 manage:app
# With RUN_MIGRATIONS=true (set on staging and production, never on PR previews,
# which share the staging database), apply database migrations before starting.
# `exec` keeps gunicorn as PID 1 so it gets SIGTERM directly on redeploy.
# One worker process with threads, as in seating and sections: Cloud Run's
# default instance is 1 vCPU, and the page polls the server, so requests are
# short and I/O-bound. Override WEB_CONCURRENCY / GUNICORN_THREADS to tune.
CMD ["sh", "-c", "if [ \"$RUN_MIGRATIONS\" = true ]; then flask db upgrade || exit 1; fi; exec gunicorn -b 0.0.0.0:${PORT:-8080} manage:app --workers ${WEB_CONCURRENCY:-1} --threads ${GUNICORN_THREADS:-8} --timeout ${GUNICORN_TIMEOUT:-120} --access-logfile -"]
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,12 @@ Then open http://localhost:3000/offerings/157/ (the course `seed_data` sets up)

`seed_data` adds sample assignments, locations, students and tickets. `python manage.py resetdb` starts the database over.

## Deployment

The app runs on Google Cloud Run, like seating and sections, from the `Dockerfile`: one non-root gunicorn worker with threads, with a health check at `/health`. The image builds the frontend with `flask build`, since in production the app serves `oh_queue/static/public/` as it is. It connects to Cloud SQL as the service account it runs as (`CLOUD_SQL_INSTANCE`, `DB_USER`, `DB_NAME`), so no database password is stored, and gets its other secrets from Secret Manager.

Each pull request from a branch in this repository is deployed as a preview on the staging project (`.github/workflows/pr-staging.yml`), and the preview's URL is commented on the PR. Previews share the staging database and never run migrations; see [docs/migrations.md](docs/migrations.md).

## Running tests

With the packages installed as above:
Expand Down
45 changes: 41 additions & 4 deletions config.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,11 +32,48 @@ def required(name, dev_default=None):
REMEMBER_COOKIE_HTTPONLY = True
REMEMBER_COOKIE_SAMESITE = "Lax"

SQLALCHEMY_DATABASE_URI = required(
"DATABASE_URL", dev_default="sqlite:///" + os.path.join(basedir, "app.db")
)

def _cloud_sql_creator(instance, user, db):
"""Connect to Cloud SQL with IAM database authentication, as the service
account the app runs as, so no database password is stored. Copied from
sections (server/config.py).

Cloud Run's built-in /cloudsql socket can't do IAM login, hence the connector.
"""
import threading

from google.cloud.sql.connector import Connector

connector = None
lock = threading.Lock()

def connect():
# Made on the first connection, not when this module is imported, so
# loading the settings doesn't look up Google credentials.
nonlocal connector
with lock:
if connector is None:
# Lazy refresh suits Cloud Run, where CPU is throttled between requests.
connector = Connector(refresh_strategy="lazy")
return connector.connect(instance, "pg8000", user=user, db=db, enable_iam_auth=True)

return connect


# The database: Cloud SQL with IAM auth (CLOUD_SQL_INSTANCE, DB_USER, DB_NAME)
# when deployed, otherwise DATABASE_URL, or in development a local SQLite file.
SQLALCHEMY_ENGINE_OPTIONS = {"pool_pre_ping": True}
CLOUD_SQL_INSTANCE = os.getenv("CLOUD_SQL_INSTANCE")
if CLOUD_SQL_INSTANCE:
SQLALCHEMY_DATABASE_URI = "postgresql+pg8000://"
SQLALCHEMY_ENGINE_OPTIONS["creator"] = _cloud_sql_creator(
CLOUD_SQL_INSTANCE, required("DB_USER"), required("DB_NAME")
)
else:
SQLALCHEMY_DATABASE_URI = required(
"DATABASE_URL", dev_default="sqlite:///" + os.path.join(basedir, "app.db")
)
SQLALCHEMY_TRACK_MODIFICATIONS = False
DATABASE_CONNECT_OPTIONS = {}

LOCAL_TIMEZONE = os.getenv("TIMEZONE", "US/Pacific")

Expand Down
20 changes: 20 additions & 0 deletions docs/migrations.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Database Migrations

The schema is managed with Flask-Migrate (Alembic); migrations live in `migrations/versions`. To change the schema, edit `oh_queue/models.py`, then from the repository root:

```
flask db migrate -m "describe the change" # generates a migration; review and edit it
flask db upgrade # applies it to your local database
```

Commit the migration with the model change. Migrations only need to work on Postgres. Deployments with `RUN_MIGRATIONS=true` (staging and production) run `flask db upgrade` when they start. PR previews never run migrations, because they share the staging database and a migration from an unmerged PR could make it unusable for staging.

## Migrations and PR Previews

As in sections, the app checks that the database's migration matches the code's latest migration (`oh_queue/schema_check.py`). While they differ, every page, API call and `/health` returns 503, rather than failing on the first query that uses a missing column. The check is repeated at most once a minute.

This means **a PR that adds a migration can't be tried on its preview.** The preview shares the staging database, which only gets the migration when the PR is merged and staging deploys, so the preview shows "This is a preview of PR #N, which adds a database migration… Merge the PR to see its changes on staging." Test such a PR locally (`flask db upgrade` against your own database), then merge it and check staging.

The reverse also happens: after another PR with a migration is merged, staging's database is ahead of a preview whose branch doesn't have that migration. The preview then says to merge main into the PR's branch, which redeploys the preview with the migration.

Locally, the same check tells you to run `flask db upgrade` when your database is behind, for example after pulling a new migration.
12 changes: 0 additions & 12 deletions entrypoint.sh

This file was deleted.

Loading
Loading