diff --git a/cspell-allow-list.txt b/cspell-allow-list.txt index a10f16fac..5bac5406b 100644 --- a/cspell-allow-list.txt +++ b/cspell-allow-list.txt @@ -440,6 +440,7 @@ refreshable registryurl reindexdb reindexing +releaseregistry relid relkind relname @@ -588,6 +589,7 @@ unmigrated unminimize untar updatecheck +upgradestore upperand urlencode urlsafe diff --git a/docs/how-to/aws-instance-sizing.mdx b/docs/how-to/aws-instance-sizing.mdx deleted file mode 100644 index dfae097ed..000000000 --- a/docs/how-to/aws-instance-sizing.mdx +++ /dev/null @@ -1,27 +0,0 @@ -# AWS Instance Sizing for Sourcegraph - - -**Deprecation notice:** AWS AMI deployments will be sunset in [Sourcegraph 7.0.0](https://sourcegraph.com/changelog/releases/7.0). We recommend migrating to [Kubernetes Helm](/self-hosted/deploy/kubernetes) or [Docker Compose](/self-hosted/deploy/docker-compose) deployment methods. Please contact support@sourcegraph.com for migration assistance. - - -## Instance size chart - -Select an instance type according to the number of users and repositories you have using this table. If you fall between two sizes, choose the larger of the two. - -For example, if you have 8,000 users with 80,000 repositories, your instance size would be **L**. If you have 1,000 users with 80,000 repositories, you should still go with size **M**. - -| | **XS** | **S** | **M** | **L** | **XL** | -| -------------------- | ----------- | ----------- | ----------- | ------------ | ------------ | -| **Users** | ≥ 500 | ≥ 1,000 | ≥ 5,000 | ≥ 10,000 | ≥ 20,000 | -| **Repositories** | ≥ 5,000 | ≥ 10,000 | ≥ 50,000 | ≥ 100,000 | ≥ 250,000 | -| **Recommended Type** | m6a.2xlarge | m6a.4xlarge | m6a.8xlarge | m6a.12xlarge | m6a.24xlarge | -| **Minimum Type** | m6a.2xlarge | m6a.2xlarge | m6a.4xlarge | m6a.8xlarge | m6a.12xlarge | - - - While we recommend certain instance types in our sizing chart, the - Sourcegraph AMI will dynamically use the resources available on the EC2 - instance type it is deployed to, provided the minimum amount of resources - needed is available. If you would like to resize your EC2 instance, follow - the [upgrade steps](/self-hosted/deploy/machine-images/aws-ami#upgrade) to switch - to a more appropriate instance type. - diff --git a/docs/self-hosted/deploy/docker-compose/configuration.mdx b/docs/self-hosted/deploy/docker-compose/configuration.mdx index 0ebdc5985..20e56a50f 100644 --- a/docs/self-hosted/deploy/docker-compose/configuration.mdx +++ b/docs/self-hosted/deploy/docker-compose/configuration.mdx @@ -1,7 +1,5 @@ # Configuration -> ⚠️ We recommend using our [machine image](/self-hosted/deploy/machine-images/), which is much easier and offers more flexibility when configuring Sourcegraph. Existing customers can reach out to our Customer Support Engineering team support@sourcegraph.com for assistance with migrating. - You can find the default base [docker-compose.yaml](https://github.com/sourcegraph/deploy-sourcegraph-docker/blob/master/docker-compose/docker-compose.yaml) file inside the [deploy-sourcegraph-docker](https://github.com/sourcegraph/deploy-sourcegraph-docker/blob/master/docker-compose) repository. We strongly recommend using an override file, instead of modifying the base docker-compose.yaml file. To configure your Sourcegraph instance, see Sourcegraph's [configuration](/admin/) docs. diff --git a/docs/self-hosted/deploy/docker-compose/migrate.mdx b/docs/self-hosted/deploy/docker-compose/migrate.mdx index 4d03d326e..cade1cf0c 100644 --- a/docs/self-hosted/deploy/docker-compose/migrate.mdx +++ b/docs/self-hosted/deploy/docker-compose/migrate.mdx @@ -2,8 +2,6 @@ The single-container (`sourcegraph/server`) deployment mode has been **removed in [Sourcegraph 7.0.0](https://sourcegraph.com/changelog/releases/7.0)**. The `sourcegraph/server` Docker image is no longer published starting with 7.0.0. You **must** migrate to Docker Compose or Kubernetes before upgrading to 7.0.0. -> ⚠️ We recommend new users use our [Kubernetes Helm](/self-hosted/deploy/kubernetes) or [Docker Compose](/self-hosted/deploy/docker-compose) deployment methods. Note: [machine image](/self-hosted/deploy/machine-images/) and [script-install](/self-hosted/deploy/single-node/script) deployments are being sunset in [Sourcegraph 7.0.0](https://sourcegraph.com/changelog/releases/7.0). Existing customers can reach out to our Customer Engineering team support@sourcegraph.com for migration assistance. - Deploying via Docker Compose is the recommended method for production deployments as it provides resource isolation between Sourcegraph services which makes it more scalable and stable. This page describes how to migrate from a single Docker image deployment to the Docker Compose deployment method. Sourcegraph's core data (including user accounts, configuration, repository-metadata, etc.), can be migrated from the single Docker image (`sourcegraph/server`) to the Docker Compose deployment by dumping and restoring the Postgres database. diff --git a/docs/self-hosted/deploy/docker-compose/operations.mdx b/docs/self-hosted/deploy/docker-compose/operations.mdx index 1e93ce31f..ba71790b8 100644 --- a/docs/self-hosted/deploy/docker-compose/operations.mdx +++ b/docs/self-hosted/deploy/docker-compose/operations.mdx @@ -1,7 +1,5 @@ # Management Operations -> ⚠️ We recommend using our [machine image](/self-hosted/deploy/machine-images/), which is much easier and offers more flexibility when configuring Sourcegraph. Existing customers can reach out to our Customer Support Engineering team support@sourcegraph.com for assistance with migrating. - --- The Sourcegraph Docker Compose yaml file uses [Docker volumes](https://docs.docker.com/storage/volumes/) to store its data. These volumes are stored at `/var/lib/docker/volumes` by [default on Linux](https://docs.docker.com/storage/#choose-the-right-type-of-mount). diff --git a/docs/self-hosted/deploy/docker-compose/upgrade.mdx b/docs/self-hosted/deploy/docker-compose/upgrade.mdx index cb41e8452..c63778255 100644 --- a/docs/self-hosted/deploy/docker-compose/upgrade.mdx +++ b/docs/self-hosted/deploy/docker-compose/upgrade.mdx @@ -68,10 +68,6 @@ $ docker-compose up -d --remove-orphans > advisement.{' '} > - -If you are upgrading to [Sourcegraph 7.0.0](https://sourcegraph.com/changelog/releases/7.0) or later and your databases are still on PostgreSQL 12, you must first upgrade to a 6.x release (such as 6.11.x) to allow the automatic PostgreSQL 12 to 16 upgrade to complete. The entrypoint script that performs this upgrade has been removed in 7.0.0. - - To perform a multi-version upgrade via migrators [upgrade](/self-hosted/updates/migrator/migrator-operations#upgrade) command on a Sourcegraph instance running on Docker compose follow the procedure below: 1. **Check Upgrade Readiness**: @@ -98,15 +94,15 @@ To perform a multi-version upgrade via migrators [upgrade](/self-hosted/updates/ ```yaml migrator: container_name: migrator - image: 'index.docker.io/sourcegraph/migrator:6.0.0' + image: 'index.docker.io/sourcegraph/migrator:{CURRENT_VERSION_NO_V}' ``` 2. Set the migrator `command:` to `upgrade` you'll need to supply a `--to=` argument to the version you're upgrading to. **Example:** ```yaml - command: ['upgrade', '--from=v5.9.0', '--to=v6.0.0'] + command: ['upgrade', '--from=v7.5.0', '--to=v8.1.0'] ``` > _Note: you may add the `--dry-run` flag to the `command:` to test things out before altering the dbs_ 3. Run migrator with `docker-compose up migrator` - - Migrator `depends_on:` will ensure the databases are ready before attempting to run the migrator. Ensuring that database entry point scripts are run before the migrator attempts to connect to the databases. For users upgrading from a version earlier than `5.10.0`, a PostgreSQL version is required and will be performed automatically here. For more details, see [Upgrading PostgreSQL](/self-hosted/postgres#upgrading-postgresql). + - Migrator `depends_on:` will ensure the databases are ready before attempting to run the migrator. Ensuring that database entry point scripts are run before the migrator attempts to connect to the databases. **Example:** ```sh $ ~/deploy-sourcegraph-docker/docker-compose/ docker-compose up migrator @@ -120,7 +116,7 @@ To perform a multi-version upgrade via migrators [upgrade](/self-hosted/updates/ ✔ Container migrator Created 0.0s ! migrator The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested 0.0s Attaching to migrator - migrator | ✱ Sourcegraph migrator 6.0.0 + migrator | ✱ Sourcegraph migrator 8.1.1335 migrator | ⚠️ Failed to check for migrator update: unexpected status code 404. Continuing... migrator | Attempting connection to frontend... migrator | ✅ Connection to frontend succeeded @@ -130,7 +126,7 @@ To perform a multi-version upgrade via migrators [upgrade](/self-hosted/updates/ migrator | ✅ Connection to codeintel succeeded migrator | Attempting connection to codeinsights... migrator | ✅ Connection to codeinsights succeeded - migrator | 👉 Migrating to v6.0 (step 1 of 1) + migrator | 👉 Migrating to v8.1 (step 1 of 1) migrator | 👉 Running schema migrations migrator | ✅ Schema migrations complete migrator exited with code 0 @@ -144,7 +140,7 @@ To perform a multi-version upgrade via migrators [upgrade](/self-hosted/updates/ ```yaml migrator: container_name: migrator - image: 'index.docker.io/sourcegraph/migrator:6.0.0@sha256:ec295eb0b743da6bf56777ca6524972267a5c442b0288095e2fe12fce38ebacc' + image: 'index.docker.io/sourcegraph/migrator:{CURRENT_VERSION_NO_V}' cpus: 0.5 mem_limit: '500m' command: ['up'] diff --git a/docs/self-hosted/deploy/kubernetes/index.mdx b/docs/self-hosted/deploy/kubernetes/index.mdx index 54b5cba34..86b0ae174 100644 --- a/docs/self-hosted/deploy/kubernetes/index.mdx +++ b/docs/self-hosted/deploy/kubernetes/index.mdx @@ -1004,21 +1004,6 @@ When all pods have restarted and show as Running, you can browse to your Sourceg **Step 2:** - - - The Kubernetes Helm deployment type does not support MVU from Sourcegraph - `v5.9.45` versions and earlier to ANY version `v6.0.x` or later. Admins - seeking to upgrade to any Sourcegraph version `v6.0.x` or later (including - `v6.1.x`, `v6.2.x`, etc.) must first upgrade to `v5.10.3940` or - `v5.11.6271` and then use the standard upgrade procedure to get to their - target version. This is because migrator in all versions from `v6.0.0` - onwards will no longer connect to Postgres 12 databases. Additionally, - starting in [Sourcegraph 7.0.0](https://sourcegraph.com/changelog/releases/7.0), the automatic PostgreSQL 12 to 16 upgrade - entrypoint script has been removed entirely. For more info see - our [PostgreSQL upgrade docs](/self-hosted/postgres#requirements). - - - Scale down `deployments` and `statefulSets` that access the database, _this step prevents services from accessing the database while schema migrations are in process._ The following services must have their replicas scaled to 0: @@ -1068,7 +1053,7 @@ $ helm upgrade --install -n --set "migrator.args={upgrade,--fro **Example:** ```sh -$ helm upgrade --install -n sourcegraph --set "migrator.args={upgrade,--from=3.41.0,--to=4.5.1}" sourcegraph-migrator sourcegraph/sourcegraph-migrator --version {CURRENT_VERSION_NO_V} +$ helm upgrade --install -n sourcegraph --set "migrator.args={upgrade,--from=7.5.0,--to=8.1.0}" sourcegraph-migrator sourcegraph/sourcegraph-migrator --version {CURRENT_VERSION_NO_V} Release "sourcegraph-migrator" has been upgraded. Happy Helming! NAME: sourcegraph-migrator LAST DEPLOYED: Tue Mar 7 18:23:56 2023 @@ -1076,8 +1061,7 @@ NAMESPACE: sourcegraph STATUS: deployed REVISION: 2 TEST SUITE: None -✅ Out of band migrations complete -👉 Migrating to v4.5 (step 3 of 3) +👉 Migrating to v8.1 (step 1 of 1) 👉 Running schema migrations ✅ Schema migrations complete ``` diff --git a/docs/self-hosted/deploy/kubernetes/upgrade.mdx b/docs/self-hosted/deploy/kubernetes/upgrade.mdx index 7ac1347bc..942e77c2b 100644 --- a/docs/self-hosted/deploy/kubernetes/upgrade.mdx +++ b/docs/self-hosted/deploy/kubernetes/upgrade.mdx @@ -20,7 +20,7 @@ A [standard upgrade](/self-hosted/updates/#upgrade-types) occurs between a Sourc ### Upgrade with Kubernetes Kustomize -The following procedure is for performing a **standard upgrade** with Sourcegraph instances version **`v4.5.0` and above**, which have [**migrated**](/self-hosted/deploy/kubernetes/kustomize/migrate) to the [deploy-sourcegraph-k8s repo](https://github.com/sourcegraph/deploy-sourcegraph-k8s). +The following procedure is for performing a **standard upgrade** with Sourcegraph instances deployed with the [deploy-sourcegraph-k8s repo](https://github.com/sourcegraph/deploy-sourcegraph-k8s). **Step 1**: Create a backup copy of the deployment configuration file @@ -111,7 +111,7 @@ To perform a multi-version upgrade via migrators [upgrade](/self-hosted/updates/ kubectl apply --prune -l deploy=sourcegraph -f cluster.yaml ``` - > Note: This step will ensure that any PostgreSQL upgrade performed as an entrypoint script will have a chance to execute before the migrator is run. For more information see [Upgrading PostgreSQL](/self-hosted/postgres#upgrading-postgresql). **Starting in [Sourcegraph 7.0.0](https://sourcegraph.com/changelog/releases/7.0), the entrypoint upgrade script has been removed. Users on PostgreSQL 12 must upgrade to a 6.x release first.** + > Note: This step will ensure that any PostgreSQL upgrade performed as an entrypoint script will have a chance to execute before the migrator is run. For more information see [Upgrading PostgreSQL](/self-hosted/postgres#upgrading-postgresql). 4. **Run Migrator with the `upgrade` command**: @@ -123,8 +123,8 @@ To perform a multi-version upgrade via migrators [upgrade](/self-hosted/updates/ ```yaml - name: migrator - image: "index.docker.io/sourcegraph/migrator:6.0.0" // here we use a later version of migrator than the version we're upgrading too. - args: ["upgrade", "--from=v5.9.0", "--to=5.11.0"] + image: "index.docker.io/sourcegraph/migrator:{CURRENT_VERSION_NO_V}" // here we use a later version of migrator than the version we're upgrading too. + args: ["upgrade", "--from=v7.5.0", "--to=v8.1.0"] env: ``` @@ -161,18 +161,8 @@ To perform a multi-version upgrade via migrators [upgrade](/self-hosted/updates/ system, terminal, and any other details, to: https://sourcegraph.com/contact - ✱ Sourcegraph migrator 4.5.1 - 👉 Migrating to v3.43 (step 1 of 3) - 👉 Running schema migrations - ✅ Schema migrations complete - 👉 Running out of band migrations [1 2 4 5 7 13 14 15 16] - ✅ Out of band migrations complete - 👉 Migrating to v4.3 (step 2 of 3) - 👉 Running schema migrations - ✅ Schema migrations complete - 👉 Running out of band migrations [17 18] - ✅ Out of band migrations complete - 👉 Migrating to v4.5 (step 3 of 3) + ✱ Sourcegraph migrator 8.1.1335 + 👉 Migrating to v8.1 (step 1 of 1) 👉 Running schema migrations ✅ Schema migrations complete $ kubectl -n sourcegraph get jobs @@ -194,14 +184,6 @@ To perform a multi-version upgrade via migrators [upgrade](/self-hosted/updates/ ``` Your instance should now be up and running in the new version! ---- - -### Using MVU to Migrate to Kustomize - -Due to limitations with the Kustomize deployment method introduced in Sourcegraph `v4.5.0`, multi-version upgrades (e.g. `v4.2.0` -> `v5.0.3`), migrations to `deploy-sourcegraph-k8s` should be conducted separately from a full upgrade. - -Admins upgrading a Sourcegraph instance older than `v4.5.0` and migrating from our [legacy kubernetes](https://github.com/sourcegraph/deploy-sourcegraph) offering to our new [kustomize manifests](https://github.com/sourcegraph/deploy-sourcegraph-k8s) should upgrade to `v4.5.0` perform the `migrate` [procedure](/self-hosted/deploy/kubernetes/kustomize/migrate) and then perform the remaining upgrade to bring Sourcegraph up to the desired version. - ## Rollback ## Rollback diff --git a/docs/self-hosted/deploy/machine-images/aws-ami.mdx b/docs/self-hosted/deploy/machine-images/aws-ami.mdx deleted file mode 100644 index 5a8b8b210..000000000 --- a/docs/self-hosted/deploy/machine-images/aws-ami.mdx +++ /dev/null @@ -1,249 +0,0 @@ -# Sourcegraph AWS AMI instances - - -**Deprecation notice:** AWS AMI and machine image deployments will be sunset in [Sourcegraph 7.0.0](https://sourcegraph.com/changelog/releases/7.0). We recommend migrating to [Kubernetes Helm](/self-hosted/deploy/kubernetes) or [Docker Compose](/self-hosted/deploy/docker-compose) deployment methods. Please contact support@sourcegraph.com for migration assistance. - - -Sourcegraph [Amazon Machine Images (AMIs)](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instances-and-amis.html) allow you to quickly deploy a production-ready Sourcegraph instance tuned to your organization's scale in just a few clicks. - -Following these docs will provision the following resources: - -- An EC2 node running Sourcegraph -- A root EBS volume with 50GB of storage -- An additional EBS volume with 500GB of storage for storing code and search indices - -### Sourcegraph AMI account ID - -All Sourcegraph AMIs are published from one of the following AWS accounts: - -- Account ID: `840044800169` -- Account ID: `870751268155` - -### Instance sizing - -Refer to our [AWS Instance Sizing](/how-to/aws-instance-sizing) guide to determine the appropriate EC2 instance type for your Sourcegraph deployment based on the number of users and repositories. - -Click [here](https://github.com/sourcegraph/deploy#amazon-ec2-amis) to see the completed list of AMI IDs published in each region. - -**The default AMI username is `ec2-user`.** - ---- - -## Deploy Sourcegraph - -![AWS Marketplace Sourcegraph](/aws-ami-marketplace.png) - -1. Search for Sourcegraph in the AWS Marketplace or AMI catalog (see screenshot in the introduction). -2. Choose **Launch instance from AMI**. -3. Name your instance. -4. Select an **instance type** according to [our sizing guide](/how-to/aws-instance-sizing) or your specific requirements. -5. **Key pair (login)**: Select or create a new Key Pair for connecting to your instance securely (this may be required in the event you need support). -6. **Network settings**: - - Under "Auto-assign public IP" select "Enable". - - Select a **Security Group** for the instance, or create one with the following rules: - - Allow SSH from Anywhere: `port range 22, source 0.0.0.0/0, ::/0` - - Allow HTTPS from the internet: `port range 443, source 0.0.0.0/0, ::/0` - - Allow HTTP traffic from the internet: `port range 80, source 0.0.0.0/0, ::/0` - - **NOTE**: If you do not wish to have HTTP/HTTPS exposed to the public internet, you may later choose to remove these rules so that all traffic routes through your AWS load balancer. -7. **Configure storage**: - - **Root Volume:** `50GB` - - **EBS Volume:** `500GB` - this should be at least 25-50% _more_ than the size of all your repositories on disk (you may check your GitHub/BitBucket/GitLab instance's disk usage.) -8. Click **Launch instance**, and navigate to the public IP address in your browser. (Look for the IPv4 Public IP value in your EC2 - instance page under the Description panel.) - -Once the instance has started, please allow ~5 minutes for Sourcegraph to initialize. During this time you may observe a `404 page not found` response. - -To configure SSL, and lock down the instance from the public internet, see the [networking](#networking) section. - - - If you cannot access the Sourcegraph homepage after 10 minutes, please try - reboot your instance. - - -### Executors - -Executors are supported using [native kubernetes executors](/self-hosted/executors/deploy-executors-kubernetes) *(Beta — not recommended for production)*. - -Executors support [auto-indexing](/code-navigation/auto-indexing) and [server-side batch changes](/batch-changes/server-side). - -To enable executors you must do the following: - -1. Connect to the AMI instance using `ssh` -2. Run `cd /home/ec2-user/deploy/install/` -3. Replace the placeholder `executor.frontendPassword` in `override.yaml` -4. Run the following command to update the executor - -``` -helm upgrade -i -f ./override.yaml --version "$(cat /home/ec2-user/.sourcegraph-version)" executor ./sourcegraph-executor-k8s-charts.tgz -``` - -1. Add the following to the advanced configuration using the password you chose previously - -``` -"executors.accessToken": "", -"executors.frontendURL": "http://sourcegraph-frontend:30080", -"codeIntelAutoIndexing.enabled": true -``` - -1. Check `Administration > Executors > Instances` to verify the executor connected successfully. If it does not appear try reboot the instance - -To use server-side batch changes, see the [server-side batch changes documentation](/batch-changes/server-side). - ---- - -## Networking - -We suggest using an AWS Application Load Balancer (ALB) to manage HTTPS connections to Sourcegraph. This makes managing SSL certificates easy. - -### Creating an AWS Load Balancer - - - You must own a domain name before you can proceed with the following steps. - - -1. Request a certificate for the domain name in [AWS Certificate Manager](https://aws.amazon.com/certificate-manager/). -2. Create a [target group](https://console.aws.amazon.com/ec2#TargetGroups) for `HTTPS Port 443` that links to the instance's `Port 443`. -3. Create a new subnet inside the instance VPC. -4. Create a new Application Load Balancer via [AWS Load Balancers](https://console.aws.amazon.com/ec2#LoadBalancers). - -#### Step 1: Request certificate - -![AWS Certificate Manager request](https://user-images.githubusercontent.com/68532117/192369850-e90d1078-7ad6-4624-acc1-db093ef4d642.png) - -Open the [AWS Certificate Manager console](https://console.aws.amazon.com/acm) to **Request a certificate**: - -- **Domain names**: Fully qualified domain name: your domain -- **Select validation method**: DNS validation—recommended - -After the certificate has been created, you will need to attach the `CNAME name` and `CNAME values` to your DNS. - -Follow the steps below to attach the CNAME to your DNS if your DNS is hosted in [AWS route 53](https://console.aws.amazon.com/route53): - -1. Click **Create record in route 53** in the certificate dashboard. -2. Select the DNS you would like to attach the certificate to. -3. Click **Create records** once you have verified the information is correct. -4. Wait ~30 minutes before the validation is completed. - -#### Step 2: Create a target group - -1. Click **Create a target group** on your [EC2 Target groups dashboard](https://console.aws.amazon.com/ec2#TargetGroups) - - Choose a target type: `Instance` - - Target group name: `YOUR-TARGETGROUP-NAME` - - Protocol: `HTTPS` - - Port: `443` - - VPC: Select the VPC where your instance is located. - - Protocol version: `HTTP2` - - Health checks: `Use Default` -2. Click **Include as pending below** - -#### Step 3: Create subnets - -Click **Create subnet** in your VPC subnets dashboard: - -- **VPC ID**: Selected the VPC that the instance is in. -- **Subnet name**: name the subnet. -- **Availability Zone**: select an availability zone that is different from the current zone. -- Click **Create subnet** - -#### Step 4: Create an Application Load Balancer - -1. Open your [EC2 Load Balancers dashboard](https://console.aws.amazon.com/ec2#LoadBalancers) to **Create Load Balancer**. -2. Choose **Application Load Balancer** as the Load balancer types using the following configurations: - -- **Basic configuration** - - Load balancer name: `YOUR-LOAD-BALANCER-NAME` - - Scheme: `Internet-facing` - - IP address type: `IPv4` -- **Network mapping** - - VPC: Selected the VPC that the instance is in. - - Mapping: Select two subnets associated with the selected VPC. -- **Security groups** - - Security groups: Make sure only the security group associated with the instance is selected. -- **Listeners and routing** - - Protocol: `HTTPS` - - Port: `443` - - Default action: Select the HTTPS target group created for the instance. - -### Securing your instance - -[Configure user authentication](/admin/auth) (SSO, SAML, OpenID Connect, etc.) to give users of your Sourcegraph instance access to it. - -Now that your instance is confirmed to be working, and you have HTTPS working through an Amazon load balancer, you may choose to secure your Sourcegraph instance further by modifying the security group/firewall rules to prevent access from the public internet. You can do this by modifying the security group/firewall rules. - ---- - -## Upgrade - - - This upgrade process works with **Sourcegraph AWS AMI instances only**. Do - not use these if you deployed Sourcegraph through other means. - - - - Please be aware that our AMI deployment types do not support multi-version - upgrades at this time. - - -Please take time to review the following before proceeding with the upgrades: - -- [Technical changelog](https://sourcegraph.com/changelog/releases) -- [Update policy](/self-hosted/updates#update-policy) -- [Update notes](https://sourcegraph.com/changelog/self-hosted/kubernetes) - -Back up your volumes before each upgrade! - -#### Step 1: Stop the current instance - -1. Stop your current Sourcegraph AMI instance - - Go to the ECS console for your instance - - Click Instance State to Stop Instance -2. Detach the non-root data volume (Device name: /dev/sdb/) - - Go to the Storage section in your instance console - - Find the volume with the device name **/dev/sdb** - - Select the volume, then click Actions to **Detach Volume** - - Give the volume a name for identification purposes -3. Make a note of the VPC name - -#### Step 2: Launch a new instance - -1. Launch a new Sourcegraph instance from an AMI with the latest version of Sourcegraph -2. Name the instance -3. Select the appropriate **instance type** -4. Under **Key Pair** - -- Select the **Key Pair** used by the old instance - -5. Under **Network settings** - - Select the **Security Group** used by the old instance -6. Under **Configure storage** - -- Remove the **second** EBS volume - -7. After reviewing the settings, click **Launch Instance** -8. Attach the detached volume to the new instance - - Go to the Volumes section in your ECS Console - - Select the volume you've detached earlier - - Click **Actions > Attach Volume** -9. On the `Attach volume` page: - -- **Instance**: select the new Sourcegraph AMI instance -- **Device name**: /dev/sdb - -10. **Reboot** the new instance - -You can terminate the stopped Sourcegraph AMI instance once you have confirmed the new instance is up and running. - -## Downgrade - -Please refer to the upgrade procedure above if you wish to roll back your instance. - ---- - -## Storage and Backups - -We strongly recommend you taking [snapshots of the entire EBS volume](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-creating-snapshot.html) on an [automatic, scheduled basis](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/snapshot-lifecycle.html). - -## Additional resources - -- [Increase the size of an Amazon EBS volume on an EC2 instance](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/modify-ebs-volume-on-instance.html) -- [Change the instance type](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-resize.html) diff --git a/docs/self-hosted/deploy/machine-images/aws-oneclick.mdx b/docs/self-hosted/deploy/machine-images/aws-oneclick.mdx deleted file mode 100644 index 61265575f..000000000 --- a/docs/self-hosted/deploy/machine-images/aws-oneclick.mdx +++ /dev/null @@ -1,132 +0,0 @@ -# AWS One-Click Installation for Sourcegraph - - -**Deprecation notice:** AWS AMI and machine image deployments will be sunset in [Sourcegraph 7.0.0](https://sourcegraph.com/changelog/releases/7.0). We recommend migrating to [Kubernetes Helm](/self-hosted/deploy/kubernetes) or [Docker Compose](/self-hosted/deploy/docker-compose) deployment methods. Please contact support@sourcegraph.com for migration assistance. - - -This page describes how to launch a verified and pre-configured Sourcegraph instance in just ~10 minutes using our one-click CloudFormation template and standard AMIs. - -Prefer manually installing on AWS yourself? See our [AMI](/self-hosted/deploy/machine-images/aws-ami) installation options. - -## Prerequisites - -- An AWS account (most regions are supported, see our [Launcher](#deploy-sourcegraph-) below for a complete list) -- General familiarity with AWS -- An [EC2 keypair](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/create-key-pairs.html) for SSH access -- (optional) A Sourcegraph license to unlock features ([view plans](https://sourcegraph.com/pricing) or [learn how licenses work](/admin/licensing/)) - -## What will be created - -Using our wizard, a single EC2 instance will be created with the following: - -- EBS root volume: 50GB -- EBS data volume: 500GB - your Sourcegraph data (code and search indices.) -- AWS Network Security Group -- The latest version of Sourcegraph -- (optional) DNS and TLS via AWS Load Balancer and AWS Certificate Manager - - - The instance will launch in the default VPC. If your AWS user does not have - a default VPC, or the option `Auto-assign public IPv4 address` is not - enabled for a subnet within that VPC, please see our [Manual - AMI](/self-hosted/deploy/machine-images/aws-ami) instructions instead. - - -![aws-oneclick](https://storage.googleapis.com/sourcegraph-assets/Docs/aws-oneclick-diagram.png) - -## Determine your instance size - -The number of users and repositories you will use with Sourcegraph determines your instance size. If you fall between two sizes, choose the larger of the two. - -Example: With 8,000 users with 80,000 repositories, your instance size would be **L**. If you have 1,000 users with 80,000 repositories, then size **M**. - -| | **XS** | **S** | **M** | **L** | **XL** | -| -------------------- | ----------- | ----------- | ----------- | ------------ | ------------ | -| **Users** | `<= 500` | `<= 1,000` | `<= 5,000` | `<= 10,000` | `<= 20,000` | -| **Repositories** | `<= 5,000` | `<= 10,000` | `<= 50,000` | `<= 100,000` | `<= 250,000` | -| **Recommended Type** | m6a.2xlarge | m6a.4xlarge | m6a.8xlarge | m6a.12xlarge | m6a.24xlarge | -| **Minimum Type** | m6a.2xlarge | m6a.2xlarge | m6a.4xlarge | m6a.8xlarge | m6a.12xlarge | - -## Deploy Sourcegraph 🎉 - - - For security, please follow [Amazon best - practices](https://docs.aws.amazon.com/accounts/latest/reference/best-practices-root-user.html) - and _do not deploy Sourcegraph using your AWS account root user. - - - - By default the CloudFormation template will create standard EBS volumes. If - you have special EBS volume encryption requirements, please see the [AWS EBS - Encryption - guide](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSEncryption.html) - and install Sourcegraph using our [Manual - AMI](/self-hosted/deploy/machine-images/aws-ami) instructions instead. - - -Choose an AWS Region in the launcher below and click **Launch Stack**. When prompted, choose your **SSH Keypair** and **Sourcegraph Instance Size** per the chart above, then **Create Stack**. - - - -### Confirm you can access Sourcegraph - -Find the URL of your Sourcegraph instance in the **Outputs** section of the AWS Stack. On first launch, Sourcegraph may take ~5 minutes to start and may display a `404 not found` page temporarily. - -### Executors - -Executors are supported using [native kubernetes executors](/self-hosted/executors/deploy-executors-kubernetes) *(Beta — not recommended for production)*. - -Executors support [auto-indexing](/code-navigation/auto-indexing) and [server-side batch changes](/batch-changes/server-side). - -To enable executors you must do the following: - -1. Connect to the AMI instance using `ssh` -2. Run `cd /home/ec2-user/deploy/install/` -3. Replace the placeholder `executor.frontendPassword` in `override.yaml` -4. Run the following command to update the executor - -```curl -helm upgrade -i -f ./override.yaml --version "$(cat /home/ec2-user/.sourcegraph-version)" executor sourcegraph/sourcegraph-executor-k8s -``` - -1. Adding the following to the advanced configuration using the password you chose previously - -```curl -"executors.accessToken": "", -"executors.frontendURL": "http://sourcegraph-frontend:30080", -"codeIntelAutoIndexing.enabled": true -``` - -1. Check `Administration > Executors > Instances` to verify the executor connected successfully. If it does not appear try reboot the instance - -To use server-side batch changes, see the [server-side batch changes documentation](/batch-changes/server-side). - -### Secure your instance - -By default Sourcegraph will be available over HTTP on the public internet. To secure it you should now perform the following: - -1. [Configure DNS and HTTPS/TLS](/self-hosted/deploy/machine-images/aws-ami#networking) using an AWS Load Balancer and AWS Certificate Manager. -2. [Configure user authentication](/admin/auth/) (SSO, SAML, OpenID Connect, etc.) -3. [Review the new Network Security Group](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html) to prevent access from the public internet and follow the principle of least privilege. - -## Managing Sourcegraph - -### Backup and restore - -We strongly recommend you taking [snapshots of the entire Sourcegraph data EBS volume](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-creating-snapshot.html) on an [automatic, scheduled basis](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/snapshot-lifecycle.html). Only the Sourcegraph data volume (500G) needs to be backed up. - -To restore from a backup, simply follow our [upgrade instructions](/self-hosted/deploy/machine-images/aws-ami#upgrade) and skip directly to **Step 2: Launch a new instance** - choosing your desired Sourcegraph version and your backed up data volume. - -### Upgrading your Sourcegraph instance - -Updates are released every month, and upgrading is a simple process: backup your instance, detach the Sourcegraph data volume, and start a new instance using the latest AMI with your data volume attached. For step-by-step instructions [see here](/self-hosted/deploy/machine-images/aws-ami#upgrade). - -### Monitoring & alerting - -Sourcegraph comes with extensive built-in monitoring dashboards & the ability to configure alerts. Please see our [monitoring guide](/self-hosted/how-to/monitoring-guide) for more information. - -### Get Support - -Feel free to reach out to `support@sourcegraph.com` if you have any questions. - -Business support, training, Slack support, SLAs, and dedicated Technical Advisors are all available through [Business and Enterprise plans](https://sourcegraph.com/pricing). diff --git a/docs/self-hosted/deploy/machine-images/gce.mdx b/docs/self-hosted/deploy/machine-images/gce.mdx deleted file mode 100644 index 34a1ef386..000000000 --- a/docs/self-hosted/deploy/machine-images/gce.mdx +++ /dev/null @@ -1,255 +0,0 @@ -# Sourcegraph Google Compute Instances - -Sourcegraph [Google Compute Engine (GCE) Images](https://cloud.google.com/compute/docs/images) allow you to quickly deploy a production-ready Sourcegraph instance tuned to your organization's scale in just a few clicks. - -Following these docs will provision the following resources: - -- A pre-configured Sourcegraph instance running on the latest version -- A root EBS volume with 50GB of storage -- An additional EBS volume for storing code and search indices - - - Connection to the internet is required to pull Docker images at first - launch. - - -## Instance size chart - -Select a deployment image and machine type according to the number of users and repositories you have using this table. - -If you fall between two sizes, choose the larger of the two. For example, if you have 8,000 users with 80,000 repositories, your instance size would be **L**. If you have 1,000 users with 80,000 repositories, you should still go with size **M**. - -| | **XS** | **S** | **M** | **L** | **XL** | -| ------------------------ | ------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | -| **Users** | greater than or equal to 500 | greater than or equal to 1,000 | greater than or equal to 5,000 | greater than or equal to 10,000 | greater than or equal to 20,000 | -| **Repositories** | greater than or equal to 5,000 | greater than or equal to 10,000 | greater than or equal to 50,000 | greater than or equal to 100,000 | greater than or equal to 250,000 | -| **Series** | N2 | N2 | N2 | N2 | N2 | -| **Recommended** | standard-8 | standard-16 | standard-32 | standard-48 | standard-96 | -| **Minimum** | standard-8 | standard-8 | standard-16 | standard-32 | standard-64 | -| **Latest Version Links** | [download](https://storage.googleapis.com/sourcegraph-images/latest/Sourcegraph-XS.tar.gz) | [download](https://storage.googleapis.com/sourcegraph-images/latest/Sourcegraph-S.tar.gz) | [download](https://storage.googleapis.com/sourcegraph-images/latest/Sourcegraph-M.tar.gz) | [download](https://storage.googleapis.com/sourcegraph-images/latest/Sourcegraph-L.tar.gz) | [download](https://storage.googleapis.com/sourcegraph-images/latest/Sourcegraph-XL.tar.gz) | - - - Sourcegraph GCE images are optimized for the specific set of resources - provided by the machine type, please ensure you use the correct GCE image - for the associated GCE machine type. You can [resize your machine type - anytime](https://cloud.google.com/compute/docs/instances/changing-machine-type-of-stopped-instance), - but your Sourcegraph GCE image must match accordingly. If needed, follow the - [upgrade steps](#upgrade) to switch to the correct GCE image that is - optimized for your machine type. - - ---- - -## Deploy Sourcegraph GCE Image - -#### Step 1: Upload the Sourcegraph instance image to your Google Cloud Storage - -![Google Cloud Storage bucket interface](https://storage.googleapis.com/sourcegraph-assets/Docs/gcp-bucket.png) - -1. Download the image (`tar.gz`) file that matches your deployment size from the [instance size chart](#instance-size-chart) -2. Navigate to [Google Cloud Storage](https://console.cloud.google.com/storage) and select a project where you will store the image and deploy the instance in -3. **Upload** the `tar.gz` file to a bucket (create one if needed) -4. Copy the **URL** - -#### Step 2: Create a new custom image - -![GCE create custom image form](https://storage.googleapis.com/sourcegraph-assets/Docs/gcp-custom-image.png) - -1. Click **CREATE IMAGE** on top of the [GCE Images Console](https://console.cloud.google.com/compute/images) - - Name: [Any] - - Source: Cloud Storage file - - Cloud Storage file: Click **Browse** and look for the image you uploaded in step 1 - - Select location: select one that works best for you - - Family: sourcegraph -2. Click **CREATE** to create your custom image of Sourcegraph - -#### Step 3: Launch a new instance - -![GCE create VM instance form](https://storage.googleapis.com/sourcegraph-assets/Docs/gcp-launch.png) - -1. Create a new VM instance from your [GCE Console](https://console.cloud.google.com/compute) -2. Name the instance -3. Select a **machine type** (see [instance size chart](#instance-size-chart)) -4. Under the **Boot disk** section, click **CHANGE** and navigate to the **CUSTOM IMAGES** tab - - Image: Select the Sourcegraph image created in the previous step - - Boot disk type: SSD persistent disk -5. Under the **Firewall** section - -- Allow both `HTTP` and `HTTPS` traffic if needed - -6. Under the **Advanced options > Disk** section - - Select **+ ADD NEW DISK** - - Disk source type: Blank disk - - Disk type: SSD persistent disk (REQUIRED) - - Size: 500GB or more --_this should be at least 25-50% *more* than the size of all your repositories on disk. Please refer to the disk usage of your code host instances (e.g. GitHub/BitBucket/GitLab) in order to adjust disk size accordingly._ - - Deletion rule: Keep disk - - - RECOMMENDED: Configure your network setting under the **Advanced options > - Network** section - - - - Please allow ~5 minutes for Sourcegraph to initialize. During this time you - may observe a `404 page not found` response. If you cannot access the - Sourcegraph homepage after 10 minutes, please try reboot your instance. - - -### Connect via SSH - -The default username for all Sourcegraph GCE image instances is: `sourcegraph`. - -To SSH into the instance as user `sourcegraph`, add the username in front of your instance name. For example: - -```bash -# sourcegraph is the username -$ gcloud compute ssh sourcegraph@YOUR-INSTANCE-NAME --zone=us-central1-a -``` - -Your can also switch users in the terminal with the command below: - -```bash -$ sudo su sourcegraph -``` - -### Executors - -Executors are supported using [native kubernetes executors](/self-hosted/executors/deploy-executors-kubernetes) *(Beta — not recommended for production)*. - -Executors support [auto-indexing](/code-navigation/auto-indexing) and [server-side batch changes](/batch-changes/server-side). - -To enable executors you must do the following: - -1. Connect to the instance using `ssh` as described above. -2. Run `cd /home/sourcegraph/deploy/install/`. -3. Replace the placeholder `executor.frontendPassword` in `override.yaml`. -4. Run the following command to update the executor: - -``` -helm upgrade -i -f ./override.yaml --version "$(cat /home/sourcegraph/.sourcegraph-version)" executor sourcegraph/sourcegraph-executor-k8s -``` - -1. Add the following to the advanced configuration using the password you chose previously: - -``` -"executors.accessToken": "", -"executors.frontendURL": "http://sourcegraph-frontend:30080", -"codeIntelAutoIndexing.enabled": true -``` - -1. Check `Administration > Executors > Instances` to verify the executor connected successfully. If it does not appear try reboot the instance. - -To use server-side batch changes, see the [server-side batch changes documentation](/batch-changes/server-side). - ---- - -## Networking - -An ephemeral external IP address is assigned to all VM instances by default; however, the IP address is not static, meaning it changes every time the VM is rebooted. - -As a result, setting up a static IP for your Sourcegraph instance is strongly recommended in production so that you can point your DNS to the static IP easily. - -## Upgrade - - - This upgrade process works with **Sourcegraph instances set up with GCE - Images only**. Do not follow this upgrade procedure if you deployed - Sourcegraph through other means. - - - - Please be aware that our machine image deployment types do not support - multi-version upgrades at this time. - - -Please take time to review the following before proceeding with the upgrades: - -- [Technical changelog](https://sourcegraph.com/changelog/releases) -- [Update policy](/self-hosted/updates#update-policy) -- [Update notes](https://sourcegraph.com/changelog/self-hosted/kubernetes) -- [Multi-version upgrade procedure](https://sourcegraph.com/changelog/self-hosted/kubernetes) - -IMPORTANT: **Back up your volumes before each upgrade** - -#### Option 1 - -SSH into the VM as user `sourcegraph` and run the following command: - -```bash -# Please update the version number according to the upgrade notes -helm upgrade -i -f /home/sourcegraph/deploy/install/override.yaml --version {CURRENT_VERSION_NO_V} sourcegraph sourcegraph/sourcegraph -# Note: /home/sourcegraph/deploy/install/override.yaml is the override file for your instance size -``` - -#### Option 2 - -##### Step 1: Stop the current instance - -![GCE instance edit button location](https://storage.googleapis.com/sourcegraph-assets/Docs/gcp-edit.png) - -1. _Stop_ your current Sourcegraph GCP Image instance from the [GCE Console](https://console.cloud.google.com/compute). -2. Navigate to your instance console after your instance is stopped. -3. Detach the local disk: - - Click the **EDIT** button located on top of the page in your instance console. - - Remove the **Existing disk** under **Additional disks**. - - **SAVE** the changes. - -##### Step 2: Launch a new instance - -1. Launch a new Sourcegraph instance from a GCE image with a newer version of Sourcegraph. -2. Name the instance. -3. Select the appropriate **machine type**. -4. Under the **Firewall** section: - -- Allow both `HTTP` and `HTTPS` traffic if needed. - -5. Under the **Advanced options > Disk** section: - -- Select **+ ATTACH EXISTING DISK**. -- Select the volume you've detached in step 1. -- Deletion rule: **Keep disk** - -6. Under the **Advanced options > Management > Automation** section: - -- Copy and paste the following script into the `Startup script` box to upgrade to a specified version. - -```bash -# Replace version number to the version you are trying to upgrade to -VERSION='4.0.0' -echo $VERSION | sudo tee home/sourcegraph/.sourcegraph-version -``` - -You can terminate the stopped Sourcegraph Google Image instance once you have confirmed the new instance is up and running. - - - You must follow the [multi-version upgrade - procedure](https://sourcegraph.com/changelog/self-hosted/kubernetes) if you - are attempting a multi-version upgrade. - - -## Downgrade - -Please refer to the upgrade procedure above if you wish to roll back your instance. The processes are identical, aside from a downgrade targeting a previous version of Sourcegraph. - ---- - -## Networking - -An ephemeral external IP address is assigned to all VM instances by default; however, the IP address is not static, meaning it changes every time the VM is rebooted. As a result, setting up a static IP for your Sourcegraph instance is strongly recommended in production so that you can point your DNS to the static IP easily. - -See the [official docs by Google on configuring IP addresses](https://cloud.google.com/compute/docs/ip-addresses/reserve-static-external-ip-address#promote_ephemeral_ip) for detailed instructions. - -Alternatively, you can set up a GCP HTTP/HTTPS Load Balancer for your Sourcegraph GCE Image instance. For more information, please refer to the [official load balancing docs by Google](https://cloud.google.com/load-balancing/docs/https), including [how to use self-managed SSL certificates](https://cloud.google.com/load-balancing/docs/ssl-certificates/self-managed-certs). - -## Storage and Backups - -We **strongly recommend** taking [snapshots of the entire data volume](https://cloud.google.com/compute/docs/disks/create-snapshots) automatically using a [snapshot schedule](https://cloud.google.com/compute/docs/disks/scheduled-snapshots). You can also set up a [snapshot schedule](https://cloud.google.com/compute/docs/disks/scheduled-snapshots) after your instance has been created. - -## Manual deploy on GCE - -Click [here](../docker-compose/google-cloud) to view install instructions for deploying on GCE manually. - -## Additional resources - -- [GCP Docs: Resize a persistent disk](https://cloud.google.com/compute/docs/disks/resize-persistent-disk) -- [GCP Docs: Changing the machine type](https://cloud.google.com/compute/docs/instances/changing-machine-type-of-stopped-instance) diff --git a/docs/self-hosted/deploy/machine-images/index.mdx b/docs/self-hosted/deploy/machine-images/index.mdx deleted file mode 100644 index 6b6f4f730..000000000 --- a/docs/self-hosted/deploy/machine-images/index.mdx +++ /dev/null @@ -1,112 +0,0 @@ -# Sourcegraph Machine Images - - -**Deprecation notice:** AWS AMI and machine image deployments will be sunset in [Sourcegraph 7.0.0](https://sourcegraph.com/changelog/releases/7.0). We recommend migrating to [Kubernetes Helm](/self-hosted/deploy/kubernetes) or [Docker Compose](/self-hosted/deploy/docker-compose) deployment methods. Please contact support@sourcegraph.com for migration assistance. - - -Sourcegraph machine images help to simplify the deployment experience for our users by using a pre-setup machine image. - -## Overview - -All Sourcegraph machine image instances are deployed via a single K3s server cluster, running on a single node with an embedded SQLite Database. It allows us to package all the Sourcegraph services with necessary components into one single launcher image so that you can spin up a Sourcegraph instance with just a few clicks in less than 10 minutes. See the official K3s docs to learn more about [the architecture of a K3s server](https://docs.k3s.io/architecture). - -This deployment method is recommended for customers who do not wish to spend too much time on setup and maintenance, but still require full control over their instances. The Sourcegraph image instances also provide high-availability and flexibility in resource usage, with the capability for scaling and making additional customizations easy whenever your needs have changed, by simply adjusting the worker/agent nodes, while still being on a single node environment. - -Everything we use to build and publish the images can be found in our [public deployment repository](https://sourcegraph.com/github.com/sourcegraph/deploy). You are also welcome to check out and follow our progress and updates there. - -Our deployment and release process is also documented in the [deployment docs](https://sourcegraph.com/github.com/sourcegraph/deploy@v4.0.1/-/blob/doc/development.md). - -All Sourcegraph machine images are free to download, and we encourage you to spin up a Sourcegraph AMI instance to experiment with. - - - While our images are free to try, you may be charged by your cloud provider - for the resources required to run the image. - - -## Features - -- Self-hosted -- Single node -- Preconfigured according to your business size -- Deployed with our [Helm Charts](https://sourcegraph.com/github.com/sourcegraph/deploy-sourcegraph-helm) to a K3s, a highly available lightweight Kubernetes distribution cluster, using the containerd run time with built-in ingress, load balancer provider, and local storage provisioner ([Click here for the full K3s dependency list](https://sourcegraph.com/github.com/k3s-io/k3s#what-is-this)) -- Easy to maintain and configure -- Supports all Sourcegraph features -- Ability to perform upgrades easily with or without SSH access -- [Executors](../executors) are not included in machine images and need to be deployed separately - - - The use of [external services](../external-services/) such as managed - Postgres databases or third party object storage services is not supported - when using Sourcegraph Machine Images. - - - - Please be aware that our machine image deployment types do not support - multi-version upgrades at this time. - - -### Amazon Machine Images (AMIs) - -All AMI instances are currently pinned with a Sourcegraph version that the instance is launched with to ensure restarting the instance will not cause upgrades accidentally. The version number is saved into a text file on both the root (file path: `/home/ec2-user/.sourcegraph-version`) and data volumes (file path: `/mnt/data/.sourcegraph-version`) where it will be read by the [reboot script](https://sourcegraph.com/github.com/sourcegraph/deploy@v5.2.5/-/blob/install/reboot.sh) on each reboot. Upgrades will only happen on reboot if the version numbers from both volumes are different. - -Detailed deployment and upgrade instructions can be found in our [AWS AMIs docs](./machine-images/aws-ami). - -Unique AMI IDs can be found in our [release page](https://github.com/sourcegraph/deploy/releases). - -#### Creation process - -Our AWS AMIs are all based on the HVM method, which provides us with the ability to create a Sourcegraph instance directly on the virtual machine using the verified `Amazon Linux 2 Kernel 5.10 x86_64 HVM` as the base image. The following steps are performed during the AMI creation process through our [installation script](https://sourcegraph.com/github.com/sourcegraph/deploy@v4.0.1/-/blob/install/install.sh): - -1. De-escalate to `ec2-user` to make sure tasks are performed by non-root user -2. Install Git -3. Clone the [deployment repository](https://github.com/sourcegraph/deploy) -4. Data volumes are formatted and labeled for the AMI instance to mount later -5. [Add configurations to the kernel](https://sourcegraph.com/github.com/sourcegraph/deploy@v4.0.1/-/blob/install/install.sh?L64-73) that would otherwise limit Sourcegraph search performance -6. [Adjust the local firewall settings](https://sourcegraph.com/github.com/sourcegraph/deploy@v4.0.1/-/blob/install/install.sh?L78-84) to ensure K3s cluster networking/DNS can pass through -7. Link the ephemeral kubelet/pod storage to our data disk -8. Link the persistent volume pod storage to our data disk -9. Link the K3s's embedded database to our data disk -10. Install K3s on root volume -11. Correct permission of the K3s kube config file located in `/etc/rancher/k3s/k3s.yaml` -12. Install Helm on root volume -13. Download Sourcegraph Helm Charts on root volume -14. Deploy Sourcegraph using the local Helm Charts -15. Save the version number to both root and data volumes -16. Add a cronjob to run the [reboot script](https://sourcegraph.com/github.com/sourcegraph/deploy@v4.0.1/-/blob/install/reboot.sh) on each reboot -17. K3s is stopped and disabled -18. The instance will then be stopped to create an AMI using the attached volumes - -#### Data volumes - -Each AWS AMI comes with two Amazon EBS volumes, one is for root, and the other one is for data: - -- The root volume contains all the files inside the sourcegraph/deploy repository that are used to build that image and deploy the Sourcegraph instance that lives inside that specific image - - File path to the deployment repo sourcegraph/deploy: `/home/ec2-user/deploy/` - - A local copy of the helm charts that were used to create the AMI: `/home/ec2-user/deploy/install/sourcegraph-charts.tar` - - Version number of the AMI Instance: `/mnt/data/.sourcegraph-version` - - Create a copy of the kube config file from `/etc/rancher/k3s/k3s.yaml` allows you to manage the k3s cluster from outside the cluster -- The data volume is where all your Sourcegraph data will be stored after the instance has been launched. The K3s embedded SQLite database is also mounted onto that volume to make back-up, upgrade, and recovery of the volumes easier. - - Data of your cluster are stored in the mounted path: `/mnt/data` - - Version number of the deployment on disk: `/mnt/data/.sourcegraph-version` - -Sourcegraph does not have access to your cluster and data. - -#### Network and Security - -![ami-diagram](https://user-images.githubusercontent.com/68532117/195904844-9257c7cd-f9b2-4d15-9c7f-a2d66a42c5df.png) - -- K3s exposes the kubelet API to the Kubernetes control plane node through a websocket tunnel in order to eliminate the need to expose extra ports on the worker nodes. -- No ingress requirements which allow cluster owners to restrict inbound traffic to only traffic within their network -- Built-in certificate rotation with the expiration date of 12 months [docs](https://docs.k3s.io/advanced) -- Enable custom certificates through [etcdctl](https://docs.k3s.io/advanced#using-etcdctl) as recommended by K3s. -- Ability to launch an AMI instance on EC2 with custom encryption [docs](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIEncryption.html#AMI-encryption-launch). - -Please refer to the official [CIS hardening guide](https://docs.k3s.io/security/hardening-guide) by K3s for more details and general tips on how to improve security of your cluster. - -> NOTE: See [Sourcegraph Vulnerability Management Policy](https://handbook.sourcegraph.com/departments/engineering/dev/policies/vulnerability-management-policy/#vulnerability-service-level-agreements) to learn more about our vulnerability and patching policy as well as the current [vulnerability service level agreements](https://handbook.sourcegraph.com/departments/engineering/dev/policies/vulnerability-management-policy/#vulnerability-service-level-agreements). - -## Additional resources - -- [sourcegraph/deploy](https://sourcegraph.com/github.com/sourcegraph/deploy) -- [Scripts used for building the AWS AMIs](https://sourcegraph.com/github.com/sourcegraph/deploy@v4.0.1/-/blob/install/install.sh) -- [Sourcegraph machine images deployment and release process](https://sourcegraph.com/github.com/sourcegraph/deploy@v4.0.1/-/blob/doc/development.md) diff --git a/docs/self-hosted/deploy/single-node/script.mdx b/docs/self-hosted/deploy/single-node/script.mdx deleted file mode 100644 index 5a79a16be..000000000 --- a/docs/self-hosted/deploy/single-node/script.mdx +++ /dev/null @@ -1,107 +0,0 @@ -# Install Sourcegraph via Shell Script - - -**Deprecation notice:** The k3s install script deployment method will be sunset in [Sourcegraph 7.0.0](https://sourcegraph.com/changelog/releases/7.0). We recommend migrating to [Kubernetes Helm](/self-hosted/deploy/kubernetes) or [Docker Compose](/self-hosted/deploy/docker-compose) deployment methods. Please contact support@sourcegraph.com for migration assistance. - - -Following these docs will launch a pre-configured single node Sourcegraph instance via shell script. - -## Supported Distros - -The script can be run on the following distros. - -- Debian -- Ubuntu -- Fedora -- RHEL -- Amazon Linux 2 - -As it is impossible to test all possible deployment environments it is possible albeit unlikely to encounter an error. We highly suggest using one of our -curated [Machine Images](/self-hosted/deploy/machine-images/) if possible. - -## Instance Size Chart - -Determine the instance type required to support the number of users and repositories you have using this table. If you fall between two sizes, choose the larger of the two. - -For example, if you have 8,000 users with 80,000 repositories, your instance size would be **L**. If you have 1,000 users with 80,000 repositories, you should go with size **M**. - -| | **XS** | **S** | **M** | **L** | **XL** | -| -------------------------- | ------------------------------ | ------------------------------- | ------------------------------- | -------------------------------- | -------------------------------- | -| **Users** | greater than or equal to 500 | greater than or equal to 1,000 | greater than or equal to 5,000 | greater than or equal to 10,000 | greater than or equal to 20,000 | -| **Repositories** | greater than or equal to 5,000 | greater than or equal to 10,000 | greater than or equal to 50,000 | greater than or equal to 100,000 | greater than or equal to 250,000 | -| **Recommended vCPU / RAM** | 8 / 32 GiB | 16 / 64 GiB | 32 / 128 GiB | 48 / 192 GiB | 96 / 384 GiB | -| **Minimum vCPU / RAM** | 8 / 32 GiB | 8 / 32 GiB | 16 / 64 GiB | 32 / 128 GiB | 48 / 192 GiB | - -## Machine Prerequisites - -Before running the install script please ensure ensure the target machine meets the following prerequisites. - -1. Sufficient CPU and RAM according to the [instance size chart](#instance-size-chart) -2. A root disk size with `50 GiB` free space for ephemeral data. -3. A secondary disk of size `250 GiB` **minimum**. Adjust to your specific needs. -4. A non-root user with `sudo` permissions to execute the script - -IMPORTANT For optimal performance both -disks should be backed by SSDs or equivalently fast storage. - -## Deploy Sourcegraph - -The script accepts the following parameters: - -#### Required - -- `-d ` - - The data disk where Sourcegraph persistent data should be stored e.g. `-d /dev/sdb` - - The script will format the disk if it has not been formatted already. - If you choose to format and mount the disk yourself it must be mounted to `/mnt/data` - MUST be a separate disk than e.g. the base OS for upgrades. - -#### Optional - -- `-s ` - - The Instance Size to use. Defaults to `XS` - -- `-v ` - - The Sourcegraph version to deploy. Defaults to the latest version - -```sh -curl -sfL https://raw.githubusercontent.com/sourcegraph/deploy/main/install/scripts/k3s/install.sh | bash -s -- -``` - -> NOTE: Please allow up to ~5 minutes for Sourcegraph to initialize - -## Networking - -Sourcegraph will be available on ports 80 and 443 (using a self signed certificate). - -We recommend deploying your own reverse proxy to terminate TLS connections with a properly signed certificate. - -## Upgrade - -- [Technical changelog](https://sourcegraph.com/changelog/releases) -- [Update policy](/self-hosted/updates#update-policy) -- [Update notes](https://sourcegraph.com/changelog/self-hosted/kubernetes) -- [Multi-version upgrade procedure](https://sourcegraph.com/changelog/self-hosted/kubernetes) - -> Important: Back up your volumes before each upgrade - -1. Shutdown the current machine running the Sourcegraph instance -2. Detach the data disk -3. Launch a new machine with the same specification as the current instance - -- Use the existing data disk in place of creating a new one - -4. Run the install script targeting the data disk. -5. Wait for Sourcegraph to start. - -## Downgrade - -Please refer to the upgrade procedure above if you wish to rollback your instance. - -## Storage and Backups - -We strongly recommend taking snapshots of the data disk on an automatic, scheduled basis. diff --git a/docs/self-hosted/index.mdx b/docs/self-hosted/index.mdx index 28e756821..eb26313f3 100644 --- a/docs/self-hosted/index.mdx +++ b/docs/self-hosted/index.mdx @@ -23,7 +23,6 @@ Get started running Sourcegraph on-prem. - [Digital Ocean installation](/self-hosted/deploy/docker-compose/digitalocean) - [Google Cloud installation](/self-hosted/deploy/docker-compose/google-cloud) - [Kubernetes](/self-hosted/deploy/kubernetes/) -- [Machine images](/self-hosted/deploy/machine-images/) - [Instance sizing](/self-hosted/deploy/instance-size) - [Resource estimator](/self-hosted/deploy/resource-estimator) - [Scaling](/self-hosted/deploy/scale) diff --git a/docs/self-hosted/postgresql-collation-version-mismatch-resolution.mdx b/docs/self-hosted/postgresql-collation-version-mismatch-resolution.mdx index abbffacf1..a09ba35a9 100644 --- a/docs/self-hosted/postgresql-collation-version-mismatch-resolution.mdx +++ b/docs/self-hosted/postgresql-collation-version-mismatch-resolution.mdx @@ -217,21 +217,6 @@ kubectl kustomize instances/my-sourcegraph -o cluster.yaml kubectl apply --prune -l deploy=sourcegraph -f cluster.yaml ``` -## AMI/Machine Image based deployments - - -**Deprecation notice:** AMI and machine image deployments will be sunset in [Sourcegraph 7.0.0](https://sourcegraph.com/changelog/releases/7.0). - - -Our AMI and GCP machine image deployments run Kubernetes internally, and thus follow the same general directions as our Helm Deployments described above. - -To access the deployment you must SSH into your AMI/Machine image first. - -- For AWS AMIs, the default username is `ec2-user` -- For GCP machine images, the default username is `sourcegraph` - -Follow the directions laid out in the Helm Deployments section. - # Expected Downtime - Reindexing may take 15-60 minutes in most cases, but depends on the database size, disk speed, and available CPU/RAM diff --git a/docs/self-hosted/updates/automatic.mdx b/docs/self-hosted/updates/automatic.mdx deleted file mode 100644 index 983931b4e..000000000 --- a/docs/self-hosted/updates/automatic.mdx +++ /dev/null @@ -1,67 +0,0 @@ -# Automatic multi-version upgrades (Deprecated) - - -**Deprecation Notice:** The autoupgrade feature has been removed in Sourcegraph 6.12.0 and later. Administrators should use the migrator [`upgrade` command](/self-hosted/updates/migrator/migrator-operations#upgrade) for multi-version upgrades instead. - -The documentation below is preserved for users on Sourcegraph versions prior to 6.12.0. - - ---- - -## Historical Documentation (Sourcegraph 5.1 - 6.11.x) - -> Warning: Automatic upgrades to v5.10.0 will fail please upgrade to a v5.9.x version and perform a standard upgrade instead! See our [postgres 12 end of life](/self-hosted/postgres12-end-of-life-notice#postgres-12-end-of-life) notice! This is one of the reasons the "Auto Upgrade" toggle is automatically turned off after a version change. This behavior is intentional—it’s designed to prevent users from unintentionally performing multi-version upgrades (MVUs) that span critical changes, such as major infrastructure updates like a PostgreSQL version upgrade. - -From **Sourcegraph 5.1 through 6.11.x**, multi-version upgrades could be performed **automatically** as if they were a standard upgrade for the same deployment type. Automatic multi-version upgrades took the following general form: - -1. Determine if your instance is ready to Upgrade: - 1. Check your Sourcegraph instances `Administration > Updates` page. ([more info](/self-hosted/updates/#upgrade-readiness)) - 2. Consult upgrade notes for your deployment type across the range of your upgrade. ([Kubernetes](https://sourcegraph.com/changelog/self-hosted/kubernetes), [Docker-compose](https://sourcegraph.com/changelog/self-hosted/docker-compose)) -2. Merge the latest Sourcegraph release into your deployment manifests. -3. With upstream changes to your manifests merged, start the new instance. - -The upgrade magic now happens when the new version is booted. In more detail, starting a new `frontend` container will: - -1. Detect that a previous version of Sourcegraph was/is currently running. -2. Plan and persist database schema migrations based on the previously running version, the new target version, and the database state. -3. Start a new internal server that sends poison pills to disconnect old services from the databases and prevents new services from connecting before the upgrade completes. -4. Start a status server in place of the `frontend` service's usual primary exposed port -5. Runs the migration plan (performing the same steps as `migration upgrade ...` ) -6. Shuts down the internal and status servers and continues to boot normally - -## Viewing progress - -During an automatic multi-version upgrade, we'll attempt to boot a status server in the frontend container that is running (or blocking) on an active upgrade attempt. If there is an upgrade failure that affects the frontend, this status page will not be available and the `frontend` container logs should be viewed. Optimistically, the status server will also be unreachable in the case that an upgrade performs quickly enough that there's no time for the status server to start. - -In the case that there's a migration failure, or an unfinished out-of-band migration that needs to be complete, the status server will be served instead of the normal Sourcegraph React app. The following screenshots show an upgrade from Sourcegraph v3.37.1 to Sourcegraph 5.0, in which the `frontend` schema is applying (or waiting to apply) a set of schema migrations, the `codeintel` schema has a pair of schema migration failures, and a single unfinished out-of-band migration is still actively being performed to completion. - -![An example in-progress upgrade with schema migrations queued for application](https://storage.googleapis.com/sourcegraph-assets/docs/images/upgrades/5.1/queued.png) -![An example in-progress upgrade with a few schema migration failures](https://storage.googleapis.com/sourcegraph-assets/docs/images/upgrades/5.1/failed.png) -![An example in-progress upgrade with unfinished out-of-band migrations](https://storage.googleapis.com/sourcegraph-assets/docs/images/upgrades/5.1/oobmigrations.png) - -## Pre v5.0.0 Automatic Multiversion Upgrades - -Whether or not a Sourcegraph instance will perform an automatic upgrade is determined by the state of the `versions` table in the `pgsql` (also called `frontend`) database. After `v5.0.0` a column exists on `versions` called `auto_upgrade` set either true of false, this controls whether or not automatic upgrades will be attempted by a Sourcegraph instance. - -For versions prior to v5.0.0, the `SRC_AUTOUPGRADE` environment variable may be set on the `sourcegraph-frontend`, and `migrator` service manifests. To enable automatic upgrades in docker-compose deployments the env var must also be set on `sourcegraph-frontend-internal`. For example, - -```yaml -sourcegraph-frontend-0: - container_name: sourcegraph-frontend-0 - image: 'index.docker.io/sourcegraph/frontend:4.5.1@sha256:22bb1203a6d8ac9bab442dcfef867efc216181026c3d6fc62415ef1a3f063139' - cpus: 4 - mem_limit: '8g' - environment: - - SRC_AUTOUPGRADE=true - - DEPLOY_TYPE=docker-compose - - 'OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4317' - - PGHOST=pgsql - - CODEINTEL_PGHOST=codeintel-db - - CODEINSIGHTS_PGDATASOURCE=postgres://postgres:password@codeinsights-db:5432/postgres - - 'SRC_GIT_SERVERS=gitserver-0:3178' - - 'SRC_SYNTECT_SERVER=http://syntect-server:9238' -``` - -## Drift and Automatic Upgrades - -Multiversion upgrades should be performed only when no drift is detected in your database, and by default if database drifts are detected the migrator `upgrade` command will not perform an upgrade. In some cases however drift may be expected, in such cases the automatic multiversion upgrade drift check may be bypassed by setting the `SRC_AUTOUPGRADE_IGNORE_DRIFT` environment variable on `sourcegraph-frontend` and `migrator` services. diff --git a/docs/self-hosted/updates/index.mdx b/docs/self-hosted/updates/index.mdx index d560849c8..01106925c 100644 --- a/docs/self-hosted/updates/index.mdx +++ b/docs/self-hosted/updates/index.mdx @@ -21,53 +21,50 @@ When upgrading Sourcegraph, it is necessary to update and apply the deployment m ### Release Schedule and versioning -Sourcegraph releases have three digits, for example: `v5.0.3` +Sourcegraph versions have three numbers, for example `v8.1.1335`: + | Description | Version | -|---------------|---------| -| major | `5` | -| minor | `0` | -| patch | `3` | +| ----------- | ------- | +| major | `8` | +| minor | `1` | +| patch | `1335` | -For more details on previous releases see [Sourcegraph Releases](/releases). +Patch numbers are not sequential. Each minor release starts at patch `0`, and later patch releases get a number based on the time elapsed since that minor release, so a higher patch number is always a newer release of the same minor version. -Generally, we ship a new minor version of Sourcegraph with new features and bug fixes monthly, followed by a patch release later that month. +Sourcegraph ships one planned release each month, usually a minor release with new features and bug fixes, and patch releases with bug fixes between planned releases as needed. Major releases replace a monthly minor release and do not follow a fixed schedule. Each major release follows the final minor release of the previous major version; for example, `v7.7` is the final `v7` minor release before `v8.0`. -Check the [Sourcegraph blog](https://about.sourcegraph.com/blog) for release announcements and details on the latest version. +Sourcegraph supports the current major version and the previous major version. For the list of supported releases, see [Sourcegraph Releases](/releases). For release notes, see the [changelog](https://sourcegraph.com/changelog/releases). ### Upgrade types -Sourcegraph has two upgrade types. **Standard** upgrades and **Multiversion** upgrades. +Sourcegraph has two upgrade types: **standard** upgrades and **multiversion** upgrades. Patch versions don't affect the upgrade type, and you should always upgrade to the latest patch of your target version. **Standard** -- Moves Sourcegraph one version forward (`v5.0.0` to `v5.1.0`), _this is usually one minor version unless the next version released is a major version_. -- Requires minor downtime in deployments except kubernetes where rolling updates are possible. +- Moves Sourcegraph to the release immediately after your current version: the next minor version (`v8.0.x` to `v8.1.x`), or the next major version from the final minor release of the previous major version (`v7.7.x` to `v8.0.x`). +- Applies database schema migrations automatically with the migrator [`up` command](/self-hosted/updates/migrator/migrator-operations#up) when the new version starts. +- Requires minor downtime, except on Kubernetes, where rolling updates are possible. **Multiversion** -- Moves Sourcegraph multiple versions forward (`v5.0.0` to `v5.2.0`). -- Requires downtime while the database schemas and rewritten and unfinished out-of-band migrations are applied -- We currently support jumping from version `v3.20` or later to any future version. -- **AMIs do not yet support multiversion upgrades. Note: AMI deployments will be sunset in 7.0.0.** - -> _Note: Patch versions don't determine upgrade type -- you should always upgrade to the latest patch._ - -| From Version | To Version | Upgrade Type | Notes | -| ------------ | ---------- | ------------ | ---------------------------------------------------- | -| `v5.0.0` | `v5.1.0` | Standard | A minor version | -| `v5.0.0` | `v5.2.0` | Multiversion | Two minor versions | -| `v3.41.0` | `v3.42.2` | Standard | A minor version and patch version | -| `v5.1.0` | `v5.1.3` | Standard | Multiple patch versions | -| `v3.43.0` | `v4.0.0` | Standard | A major version change but only one absolute version | -| `v5.0.0` | `v5.2.2` | Multiversion | Two minor versions and patch | -| `v3.33.0` | `v5.2.0` | Multiversion | Major and minor | -| `v4.5.0` | `v5.0.0` | Standard | This is a major version but only one version change | -| `v4.4.2` | `v5.0.3` | Multiversion | Major, minor, and patch | +- Moves Sourcegraph more than one release forward (`v7.5.x` to `v8.1.x`). +- Uses the migrator [`upgrade` command](/self-hosted/updates/migrator/migrator-operations#upgrade). The `frontend` service and the migrator `up` command refuse to start when an upgrade crosses more than one minor version. +- Requires downtime while migrator applies database schema migrations and completes unfinished out-of-band migrations. +- Supports upgrading from any [supported release](/releases) to the latest version. If you run an older version, contact [support@sourcegraph.com](mailto:support@sourcegraph.com) before you upgrade. + +| From Version | To Version | Upgrade Type | Notes | +| ------------ | ----------- | ------------ | ------------------------------------------------------ | +| `v8.1.0` | `v8.1.1335` | Standard | A patch release | +| `v8.0.x` | `v8.1.x` | Standard | One minor version | +| `v7.7.x` | `v8.0.x` | Standard | The final `v7` minor release to the next major release | +| `v7.5.x` | `v7.7.x` | Multiversion | Two minor versions | +| `v7.6.x` | `v8.0.x` | Multiversion | `v7.6` is not the final `v7` minor release | +| `v7.5.x` | `v8.1.x` | Multiversion | A major version and multiple minor versions | > _Note:_ > -> - _Our major releases do not occur on a consistent interval, so make sure to check our changelog if you aren't certain about whether a major version is multiple minor versions away from your current version. You can also reach out to our support team [support@sourcegraph.com](mailto:support@sourcegraph.com)_ -> - _Sourcegraph guarantees database backward compatibility to the most recent minor version._ +> - _If you aren't sure how many releases separate your current and target versions, check the [changelog](https://sourcegraph.com/changelog/releases) or contact [support@sourcegraph.com](mailto:support@sourcegraph.com)._ +> - _Database schemas stay backward compatible with the previous minor version, so you can roll back one minor version without rolling back your databases._ ### Sourcegraph databases & migrator @@ -125,15 +122,10 @@ If your instance has schema drift or unfinished oob migrations you may need to a - [Standard Upgrade Operations](/self-hosted/deploy/kubernetes#standard-upgrades) - [Multiversion Upgrade Operations](/self-hosted/deploy/kubernetes#multi-version-upgrades) - [Upgrade Notes](https://sourcegraph.com/changelog/self-hosted/kubernetes) -- **Single-container Sourcegraph with Docker (removed in 7.0.0)** - - The single-container deployment mode has been sunset. [Migrate to Docker Compose](/self-hosted/deploy/docker-compose/migrate) or [Kubernetes](/self-hosted/deploy/kubernetes/). -- [**Sourcegraph AWS AMI instances**](/self-hosted/deploy/machine-images/aws-ami#upgrade) *(deprecated — sunset in 7.0.0)* ## Other helpful links - [PostgreSQL Collation Version Mismatch Resolution](/self-hosted/postgresql-collation-version-mismatch-resolution) -- [Postgres 12 End Of Life Notice](/self-hosted/postgres12-end-of-life-notice) — **Note:** Starting in 7.0.0, the automatic PG12→PG16 upgrade entrypoint script has been removed. Users on PG12 must upgrade to a 6.x release first. - [Migrator operations](/self-hosted/updates/migrator/migrator-operations) -- [Upgrading Early Versions](/self-hosted/updates/migrator/upgrading-early-versions) - [Troubleshooting upgrades](/self-hosted/updates/migrator/troubleshooting-upgrades) - [Downgrading](/self-hosted/updates/migrator/downgrading) diff --git a/docs/self-hosted/updates/migrator/downgrading.mdx b/docs/self-hosted/updates/migrator/downgrading.mdx index f1fc219a7..9491ab933 100644 --- a/docs/self-hosted/updates/migrator/downgrading.mdx +++ b/docs/self-hosted/updates/migrator/downgrading.mdx @@ -25,25 +25,20 @@ Heres an example of `downgrade` being used on a Docker-compose deployment: _Altered `command:` value in the `docker-compose.yaml`_ ```yaml -command: ['downgrade', '--from=v4.5.1', '--to=v4.0.0'] +command: ['downgrade', '--from=v8.1', '--to=v8.0'] ``` _Command and output_ ```bash -$ ~/deploy-sourcegraph-docker/docker-compose/ v4.5.1* docker-compose up -d migrator +$ ~/deploy-sourcegraph-docker/docker-compose/ v8.1.1335* docker-compose up -d migrator codeinsights-db is up-to-date codeintel-db is up-to-date pgsql is up-to-date Recreating migrator ... done -$ ~/deploy-sourcegraph-docker/docker-compose/ v4.5.1* docker logs migrator -✱ Sourcegraph migrator 5.0.3 -👉 Migrating to v4.3 (step 1 of 2) -👉 Running schema migrations -✅ Schema migrations complete -👉 Running out of band migrations [17 18] -✅ Out of band migrations complete -👉 Migrating to v4.0 (step 2 of 2) +$ ~/deploy-sourcegraph-docker/docker-compose/ v8.1.1335* docker logs migrator +✱ Sourcegraph migrator 8.1.1335 +👉 Migrating to v8.0 (step 1 of 1) 👉 Running schema migrations ✅ Schema migrations complete ``` diff --git a/docs/self-hosted/updates/migrator/index.mdx b/docs/self-hosted/updates/migrator/index.mdx index f52ef1dcd..1f8e4c148 100644 --- a/docs/self-hosted/updates/migrator/index.mdx +++ b/docs/self-hosted/updates/migrator/index.mdx @@ -4,4 +4,3 @@ - [Migrator operations](/self-hosted/updates/migrator/migrator-operations) - [Schema drift explained](/self-hosted/updates/migrator/schema-drift) - [Troubleshooting Upgrades](/self-hosted/updates/migrator/troubleshooting-upgrades) -- [Upgrading Early Versions](/self-hosted/updates/migrator/upgrading-early-versions) diff --git a/docs/self-hosted/updates/migrator/migrator-operations.mdx b/docs/self-hosted/updates/migrator/migrator-operations.mdx index 859b1a382..211a37b66 100644 --- a/docs/self-hosted/updates/migrator/migrator-operations.mdx +++ b/docs/self-hosted/updates/migrator/migrator-operations.mdx @@ -27,9 +27,8 @@ Whenever a `docker-compose` or `kubernetes` based deployment is started `migrato Some notes on `migrator`: -- The `migrator` service was introduced in `v3.37.0`. -- `migrator`'s `upgrade` command was introduced in `v4.0.0` and is intended to be used to upgrade versions as old as `3.20.x`. **Always check your deployment types [upgrade notes](/self-hosted/updates/#upgrades-index) before a multiversion upgrade.** -- Commands such as `downgrade`, `drift`, `run-out-of-band-migrations`, and `upgrade`, all work against Sourcegraph versions as old as `v3.20.x`. +- The `upgrade` and `downgrade` commands in the latest `migrator` image work across all [supported releases](/releases). +- **Always check your deployment types [upgrade notes](/self-hosted/updates/#upgrades-index) before a multiversion upgrade.** ## Migrator environment variables @@ -84,8 +83,8 @@ upgrade \ **Required arguments**: -- `--from`: The current Sourcegraph release version (_without the patch_; e.g., `v3.36`) -- `--to`: The target Sourcegraph release version (_without the patch_; e.g., `v4.0`) +- `--from`: The current Sourcegraph release version (_without the patch_; e.g., `v7.5`) +- `--to`: The target Sourcegraph release version (_without the patch_; e.g., `v8.1`) **Optional arguments**: @@ -122,7 +121,7 @@ drift \ Exactly one of `--version` and `--file` must be supplied. -- `--version`: The instance's current Sourcegraph release version _including a patch_ (e.g., `v3.42.1`). +- `--version`: The instance's current Sourcegraph release version _including a patch_ (e.g., `v8.1.1335`). - `--file`: The filepath to a local schema description file. This is useful for airgapped instances that do not have access to the public Sourcegraph GitHub repository or the public GCS bucket where old revisions have been backfilled. - `--ignore-migrator-update`: Controls whether to hard- or soft-fail if a newer migrator version is available. It is recommended to use the latest migrator version. @@ -142,8 +141,8 @@ downgrade \ **Required arguments**: -- `--from`: The current Sourcegraph release version (_without the patch_; e.g., `v3.36`) -- `--to`: The target Sourcegraph release version (_without the patch_; e.g., `v4.0`) +- `--from`: The current Sourcegraph release version (_without the patch_; e.g., `v8.1`) +- `--to`: The target Sourcegraph release version (_without the patch_; e.g., `v8.0`) **Optional arguments**: @@ -298,8 +297,8 @@ For example here the `upgrade` command: ```yaml containers: - name: migrator - image: 'index.docker.io/sourcegraph/migrator:5.0.3' - args: ['upgrade', '--from=v3.41.0', '--to=v4.5.1'] + image: 'index.docker.io/sourcegraph/migrator:{CURRENT_VERSION_NO_V}' + args: ['upgrade', '--from=v7.5', '--to=v8.1'] envFrom: ``` @@ -326,10 +325,10 @@ Running migrator operations in helm takes advantage of the [sourcegraph-migrator Generally the migrator is run via the `helm upgrade`, for example: ``` -$ helm upgrade --install -n sourcegraph --set "migrator.args={drift,--db=frontend,--version=v3.39.1}" sourcegraph-migrator sourcegraph/sourcegraph-migrator --version 4.4.2 +$ helm upgrade --install -n sourcegraph --set "migrator.args={drift,--db=frontend,--version=v8.1.1335}" sourcegraph-migrator sourcegraph/sourcegraph-migrator --version {CURRENT_VERSION_NO_V} ``` -In the example above the `drift` operation is run with flags `--db` and `--version`. The migrator is run using image version `v4.4.2`. +In the example above the `drift` operation is run with flags `--db` and `--version`. The migrator is run using the chart version for the latest Sourcegraph release. Arguments are set with the `--set "migrator.args={operation-arg,flag-arg-1,flag-arg-2}` portion of the command. Just like you would run commands in terminal, these are the args you are telling the migrator to run on initialization. @@ -354,10 +353,10 @@ For example here the `upgrade` command: ```yaml migrator: container_name: migrator - image: 'index.docker.io/sourcegraph/migrator:5.0.3' + image: 'index.docker.io/sourcegraph/migrator:{CURRENT_VERSION_NO_V}' cpus: 0.5 mem_limit: '500m' - command: ['upgrade', '--from=v3.41.0', '--to=v4.5.1'] + command: ['upgrade', '--from=v7.5', '--to=v8.1'] environment: ``` diff --git a/docs/self-hosted/updates/migrator/upgrading-early-versions.mdx b/docs/self-hosted/updates/migrator/upgrading-early-versions.mdx deleted file mode 100644 index 601e32b70..000000000 --- a/docs/self-hosted/updates/migrator/upgrading-early-versions.mdx +++ /dev/null @@ -1,49 +0,0 @@ -# Upgrading Early Versions - -This document provides advise on multiversion upgrades starting from `v4.0.0` and earlier. - -Reminders: - -- Except in the case of the `up` command, always run `migrator` commands with the most recent release image. -- Always refer to the [upgrade notes](/self-hosted/updates/#upgrades-index) for the versions you'll pass over in the case of a multiversion upgrade. - -## Before v4.0.0 - -The `upgrade` `migrator` command is introduced in `v4.0.0`, just remember to use the latest version of the `migrator` image released during a multiversion upgrade and you'll have access to the most up to date `migrator` commands. - -## Before v3.37.0 - -In `v3.37.0` the `migrator` service was introduced. Docker-compose and Kubernetes instances hoping to upgrade from versions before the introduction of migrator will need to create the `migrator` manifest in order to run `migrator` commands. - -- In Docker-compose add the [migrator service](https://github.com/sourcegraph/deploy-sourcegraph-docker/blob/master/docker-compose/docker-compose.yaml#LL3C1-L53C35) to the top of your `docker-compose.yaml` file and follow the [multiversion upgrade procedure](/self-hosted/deploy/docker-compose/upgrade#multi-version-upgrades). -- In kubernetes deployments you'll need to create the [`migrator.Job.yaml` job manifest](https://github.com/sourcegraph/deploy-sourcegraph/blob/master/configure/migrator/migrator.Job.yaml) and use it as described in the [multiversion upgrade procedure](/self-hosted/deploy/kubernetes/upgrade#multi-version-upgrades). - > Note: _It may be more effective in some cases to `git checkout` a later version of Sourcegraph to get access to `migrator` manifests and invoke migrator this was before checking back out your `release` branch and proceeding with the upgrade._ - -## Before v3.27.0 - -In version `3.27` `pgsql` and `codeintel-db` databases were upgraded from Postgres 11 to Postgres 12. **If upgrading from 3.26 or before to 3.27 or later**, the `pgsql` and `codeintel-db` databases must have their Postgres version upgraded. If this step is not performed, then the following upgrade procedure will fail fast (and leave all existing data untouched). - -- If using an external database, follow the [upgrading external PostgreSQL instances](/self-hosted/postgres#upgrading-external-postgresql-instances) guide. -- Otherwise, perform the following steps from the [upgrading internal Postgres instances](/self-hosted/postgres#upgrading-internal-postgresql-instances) guide. - -The following procedures describe how to upgrade the Postgres version: - -### Docker-compose postgres version upgrade - -1. It's assumed that your fork of `deploy-sourcegraph-docker` is up to date with your instance's current version. Pull the upstream changes for `v3.27.0` and resolve any git merge conflicts. We need to temporarily boot the containers defined at this specific version to rewrite existing data to the new Postgres 12 format. -2. Run `docker-compose up pgsql` to launch new Postgres 12 containers and rewrite the old Postgres 11 data. This may take a while, but streaming container logs should show progress. -3. Wait until the database container is accepting connections. Once ready, run the command `docker exec pgsql -- psql -U sg -c 'REINDEX database sg;'` to repair indexes that were silently invalidated by the previous data rewrite step. **If you skip this step**, then some data may become inaccessible under normal operation, the following steps are not guaranteed to work, and **data loss will occur**. -4. Follow the same steps for the `codeintel-db`: - - Run `docker-compose up codeintel-db` to launch Postgres 12. - - Run `docker exec codeintel-db -- pgsql -U sg -c 'REINDEX database sg;'` to reindex the database. - -### Kubernetes postgres version upgrade - -1. It's assumed that your fork of `deploy-sourcegraph` is up to date with your instance's current version. Pull upstream changes for `v3.27.0` and resolve any git merge conflicts. We need to temporarily boot the containers defined in this specific version to rewrite existing data to the new Postgres 12 format. -2. Run `kubectl apply -l deploy=sourcegraph -f base/pgsql` to launch a new Postgres 12 container and rewrite the Postgres 11 data. This may take a while, but streaming container logs should show progress. - > **NOTE**: _The Postgres migration requires enough capacity in its attached volume to accommodate an additional copy of the data currently on disk. Resize the volume now if necessary—the container will fail to start if there is not enough free disk space._ -3. Wait until the database container is accepting connections. Once ready, run the command `kubectl exec pgsql -- psql -U sg -c 'REINDEX database sg;'` issue a reindex command to Postgres to repair indexes that were silently invalidated by previous data rewrite step. **If you skip this step**, then some data may become inaccessible under normal operation, following steps are not guaranteed to work, and **data loss will occur**. -4. Follow the same steps for the `codeintel-db`: - - Run `kubectl apply -l deploy=sourcegraph -f base/codeintel-db` to launch Postgres 12. - - Run `kubectl exec codeintel-db -- psql -U sg -c 'REINDEX database sg;'` to issue a reindex command to Postgres, -5. Leave these versions of the databases running while the subsequent migration steps are performed. If `codeinsights-db` is a container new to your instance, now is a good time to start it as well. diff --git a/src/data/navigation.ts b/src/data/navigation.ts index c56c1d971..e224d2b72 100644 --- a/src/data/navigation.ts +++ b/src/data/navigation.ts @@ -595,10 +595,6 @@ export const navigation: NavigationItem[] = [ { title: 'Sourcegraph on Kubernetes with Helm', href: '/self-hosted/deploy/kubernetes' - }, - { - title: 'Sourcegraph machine images', - href: '/self-hosted/deploy/machine-images' } ] }, diff --git a/src/data/redirects.ts b/src/data/redirects.ts index 4c1727167..e6cfbc2e2 100644 --- a/src/data/redirects.ts +++ b/src/data/redirects.ts @@ -3275,19 +3275,39 @@ const redirectsData = [ }, { source: '/admin/deploy/machine-images/aws-ami', - destination: '/self-hosted/deploy/machine-images/aws-ami' + destination: '/self-hosted/deploy' + }, + { + source: '/self-hosted/deploy/machine-images/aws-ami', + destination: '/self-hosted/deploy' }, { source: '/admin/deploy/machine-images/aws-oneclick', - destination: '/self-hosted/deploy/machine-images/aws-oneclick' + destination: '/self-hosted/deploy' + }, + { + source: '/self-hosted/deploy/machine-images/aws-oneclick', + destination: '/self-hosted/deploy' }, { source: '/admin/deploy/machine-images/gce', - destination: '/self-hosted/deploy/machine-images/gce' + destination: '/self-hosted/deploy' + }, + { + source: '/self-hosted/deploy/machine-images/gce', + destination: '/self-hosted/deploy' + }, + { + source: '/how-to/aws-instance-sizing', + destination: '/self-hosted/deploy/instance-size' }, { source: '/admin/deploy/machine-images', - destination: '/self-hosted/deploy/machine-images' + destination: '/self-hosted/deploy' + }, + { + source: '/self-hosted/deploy/machine-images', + destination: '/self-hosted/deploy' }, { source: '/admin/deploy/migrate-backup', @@ -3311,7 +3331,11 @@ const redirectsData = [ }, { source: '/admin/deploy/single-node/script', - destination: '/self-hosted/deploy/single-node/script' + destination: '/self-hosted/deploy/single-node' + }, + { + source: '/self-hosted/deploy/single-node/script', + destination: '/self-hosted/deploy/single-node' }, { source: '/admin/deploy/without_service_discovery', @@ -3552,7 +3576,11 @@ const redirectsData = [ }, { source: '/admin/updates/automatic', - destination: '/self-hosted/updates/automatic' + destination: '/self-hosted/updates' + }, + { + source: '/self-hosted/updates/automatic', + destination: '/self-hosted/updates' }, { source: '/admin/updates/docker_compose', @@ -3589,7 +3617,11 @@ const redirectsData = [ }, { source: '/admin/updates/migrator/upgrading-early-versions', - destination: '/self-hosted/updates/migrator/upgrading-early-versions' + destination: '/self-hosted/updates' + }, + { + source: '/self-hosted/updates/migrator/upgrading-early-versions', + destination: '/self-hosted/updates' }, { source: '/admin/updates/pure_docker',