云原生之旅 - 4)基础设施即代码 使用 Terraform 创建 Kubernetes( 二 )


变量 locals.tf
master_authorized_networks 需要改为自己要放行的白名单,只有白名单的IP才能访问 cluster api endpoint 。为了安全性,不要用0.0.0.0/0
locals {# project detailsproject = {project_id= "sre-eng-cn-dev"region= "us-central1"network_name= "wade-test-network"}# cluster detailswade_cluster = {cluster_name= "wade-gke"cluster_version= "1.22.12-gke.500"subnet_name= "wade-gke"subnet_range= "10.254.71.0/24"secondary_ip_range_pods= "172.20.72.0/21"secondary_ip_range_services = "10.127.8.0/24"region= "us-central1"node_pools = [{name= "app-pool"machine_type= "n1-standard-2"node_locations= join(",", slice(data.google_compute_zones.available.names, 0, 3))initial_node_count = 1min_count= 1max_count= 10max_pods_per_node= 64disk_size_gb= 100disk_type= "pd-standard"image_type= "COS"auto_repair= trueauto_upgrade= falsepreemptible= falsemax_surge= 1max_unavailable= 0}]node_pools_labels = {all = {}}node_pools_tags = {all = ["k8s-nodes"]}node_pools_metadata = https://www.huyubaike.com/biancheng/{all = {disable-legacy-endpoints ="true"}}node_pools_taints = {all = []}oauth_scopes = {all = ["https://www.googleapis.com/auth/monitoring","https://www.googleapis.com/auth/compute","https://www.googleapis.com/auth/devstorage.full_control","https://www.googleapis.com/auth/logging.write","https://www.googleapis.com/auth/service.management","https://www.googleapis.com/auth/servicecontrol",]}master_authorized_networks = [{display_name = "Whitelist 1"cidr_block= "4.14.xxx.xx/32"},{display_name = "Whitelist 2"cidr_block= "64.124.xxx.xx/32"},]}} output.tf output "cluster_id" {description = "GKE cluster ID"value= https://www.huyubaike.com/biancheng/module.wade-gke.cluster_id}output"cluster_endpoint" {description = "Endpoint for GKE control plane"value= https://www.huyubaike.com/biancheng/module.wade-gke.endpointsensitive= true}output"cluster_name" {description = "Google Kubernetes Cluster Name"value= https://www.huyubaike.com/biancheng/module.wade-gke.name}output"region" {description = "GKE region"value= https://www.huyubaike.com/biancheng/module.wade-gke.region}output"project_id" {description = "GCP Project ID"value= https://www.huyubaike.com/biancheng/local.project.project_id}tf文件结构如下

云原生之旅 - 4)基础设施即代码 使用 Terraform 创建 Kubernetes

文章插图
部署确保自己的GCP account已经登陆,并且有足够的权限操作GCP Project 。
gcloud auth logingcloud auth listterraform initterraform planterraform apply配置连接GKE集群【云原生之旅 - 4)基础设施即代码 使用 Terraform 创建 Kubernetes】### Adding the cluster to your contextgcloud container clusters get-credentials $(terraform output -raw cluster_name) \--region $(terraform output -raw region) \--project $(terraform output -raw project_id)
使用 下载安装 kubectl 来管理以及部署资源到集群 。  ### 本文首发于博客园 https://www.cnblogs.com/wade-xu/p/16839468.html 使用Terraform创建EKS准备好S3 bucket,更新backendproviders.tfterraform {backend "s3" {bucket = "sre-dev-terraform"key= "test/eks.tfstate"region = "cn-north-1"}required_providers {aws = {source= "hashicorp/aws"version = "~> 4.25.0"}}}provider "aws" {region= local.region}# https://github.com/terraform-aws-modules/terraform-aws-eks/issues/2009provider "kubernetes" {host= module.wade-eks.cluster_endpointcluster_ca_certificate = base64decode(module.wade-eks.cluster_certificate_authority_data)exec {api_version = "client.authentication.k8s.io/v1beta1"command= "aws"# This requires the awscli to be installed locally where Terraform is executedargs = ["eks", "get-token", "--cluster-name", module.wade-eks.cluster_id]}}
类似的,使用terraform aws module,这里有个小插曲,我建的时候提示 cn-north-1d 这个zone没有足够的资源,所以我在data available zone 里面排除了这个zone
data "aws_availability_zones" "available" {# Cannot create cluster because cn-north-1d,# the targeted availability zone, does not currently have sufficient capacity to support the cluster.exclude_names = ["cn-north-1d"]}module "wade-eks" {source= "terraform-aws-modules/eks/aws"version = "18.27.1"cluster_name= local.cluster_namecluster_version = local.cluster_versioncluster_endpoint_private_access = truecluster_endpoint_public_access= true# api server authorized network listcluster_endpoint_public_access_cidrs = local.master_authorized_networkscluster_addons = {coredns = {resolve_conflicts = "OVERWRITE"}kube-proxy = {}vpc-cni = {resolve_conflicts = "OVERWRITE"}}vpc_id= module.vpc.vpc_idsubnet_ids = module.vpc.private_subnets# Extend cluster security group rulescluster_security_group_additional_rules = local.cluster_security_group_additional_ruleseks_managed_node_group_defaults = {ami_type= local.node_group_default.ami_typemin_size= local.node_group_default.min_sizemax_size= local.node_group_default.max_sizedesired_size = local.node_group_default.desired_size}eks_managed_node_groups = {# dmz = {#name = "dmz-pool"# }app = {name= "app-pool"instance_types= local.app_group.instance_typescreate_launch_template= falselaunch_template_name= ""disk_size= local.app_group.disk_sizecreate_security_group= truesecurity_group_name= "app-node-group-sg"security_group_use_name_prefix = falsesecurity_group_description= "EKS managed app node group security group"security_group_rules= local.app_group.security_group_rulesupdate_config = {max_unavailable_percentage = 50}}}# aws-auth configmap# create_aws_auth_configmap = truemanage_aws_auth_configmap = trueaws_auth_roles = [{rolearn= "arn:aws-cn:iam::9935108xxxxx:role/CN-SRE" # replace meusername = "sre"groups= ["system:masters"]},]aws_auth_users = [{userarn= "arn:aws-cn:iam::9935108xxxxx:user/wadexu" # replace meusername = "wadexu"groups= ["system:masters"]},]tags = {Environment = "dev"Terraform= "true"}# aws china only because https://github.com/terraform-aws-modules/terraform-aws-eks/pull/1905cluster_iam_role_dns_suffix = "amazonaws.com"}

经验总结扩展阅读