AWS migration and multi-account networking
Moving a company's on-premises services to AWS with no downtime, on a multi-account network built as code with Pulumi and Terraform.
- AWS
- Pulumi
- Terraform
- Packer
- Ansible
Architecture
- Office → Site-to-site VPN: IPsec
- Site-to-site VPN → Transit Gateway
- Transit Gateway → VPCs: attachments
- VPCs → ALB
- ALB → EC2: HTTPS
- EC2 → S3
- Route 53 → ALB: alias
- Office: Team
- Transit Gateway: Shared via AWS RAM
- Route 53: Internal DNS
- VPCs: Several per account
- EC2: GitLab · Jenkins · Harbor · Artifactory
- S3: Artifacts and backups
- office: Office
- aws · shared network account: Site-to-site VPN, Transit Gateway, Route 53
- aws · workload accounts: VPCs, ALB, EC2, S3
- On-premises VM → Packer: rebuild
- Packer → EC2: AMI
- Ansible → EC2: configure
- Pulumi → EC2
- Terraform → EC2
- Route 53 → EC2: switch DNS
- On-premises VM: Before
- Packer: Image
- Ansible: Configuration
- EC2: After
- Route 53: Cutover off-hours
- Pulumi: Infrastructure
- Terraform: Infrastructure
- on-premises: On-premises VM
- build: Packer, Ansible, Pulumi, Terraform
- aws: EC2, Route 53
Context
The company ran its core engineering services (GitLab, Jenkins, Harbor and Artifactory) on on-premises virtual machines. Every team depends on them every day.
Problem
Move them to AWS without stopping anyone’s work, and end up with a network that several accounts and projects can share safely.
Approach
I migrated in order of impact, from the least critical services to the most critical, so each step was proven before the next. Each service was rebuilt on EC2 from an image baked with Packer and configured with Ansible, and every cutover happened outside working hours.
The network is defined as code: several VPCs per account, joined by a Transit Gateway shared through AWS RAM, with VPC peering where it fits and a site-to-site VPN for the office. The services sit behind Application Load Balancers, with DNS on Route 53 and storage on S3.
Outcome
The first four core services (GitLab, Jenkins, Harbor and Artifactory) have moved with no downtime, and the remaining services follow the same path. Along the way I cut the company’s AWS spend by about 30%.