AWS EKS 集群部署 KubeSphere

发表于 更新于

AWS

创建角色

1.IAM 新建角色

说明 值
服务 ec2
名称 admin 
1
2
3
4
5
6
7
8
9
10
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "*",
            "Resource": "*"
        }
    ]
}

2.新建EC2

说明 值
系统 Amazon Linux
附加角色 admin

3.登录系统,部署环境

kubectl

下载

1
curl -O https://s3.us-west-2.amazonaws.com/amazon-eks/1.27.1/2023-04-19/bin/linux/amd64/kubectl

授权

1
chmod +x ./kubectl

环境变量

1
mkdir -p $HOME/bin && cp ./kubectl $HOME/bin/kubectl && export PATH=$PATH:$HOME/bin

查看当前版本

1
kubectl version --short --client

eksctl

1
2
3
4
5
curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp

sudo mv /tmp/eksctl /usr/local/bin

eksctl version

helm

1
2
3
4
5
wget https://get.helm.sh/helm-v3.11.1-linux-amd64.tar.gz

tar -zxvf helm-v3.11.1-linux-amd64.tar.gz

sudo mv linux-amd64/helm /usr/local/bin/helm

AWS - 环境变量

  • 区域配置

    1
    2
    3
    4
    5
    export AWS_REGION=ap-south-2

    echo "export AWS_REGION=${AWS_REGION}" | tee -a ~/.bash_profile

    aws configure set default.region ${AWS_REGION}

  • 账号配置

    1
    2
    3
    4
    5
    export ACCOUNT_ID=812246909648

    echo "export ACCOUNT_ID=${ACCOUNT_ID}" | tee -a ~/.bash_profile

    export CLUSTER_NAME="eks"
    说明 值
    ACCOUNT_ID AWS 账号ID,AWS 页面右上角
    CLUSTER_NAME 集群名称

  • 集群配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
cat << EOF > ${CLUSTER_NAME}-cluster.yaml
---
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
  name: ${CLUSTER_NAME} # EKS Cluster name
  region: ${AWS_REGION} # Region Code to place EKS Cluster
  version: "1.26"
  tags:
    karpenter.sh/discovery: ${CLUSTER_NAME}
vpc:
  cidr: "10.101.0.0/16" # CIDR of VPC for use in EKS Cluster
  nat:
    gateway: Disable # Disable NAT Gateway
karpenter:
  version: 'v0.30.0'
  createServiceAccount: true # default is false
  defaultInstanceProfile: 'KarpenterNodeInstanceProfile' # default is to use the IAM instance profile created by eksctl
  withSpotInterruptionQueue: true # adds all required policies and rules for supporting Spot Interruption Queue, default is false
iam:
  withOIDC: true # required
managedNodeGroups:
  - name: node-group # Name of node group in EKS Cluster
    instanceType: t3.xlarge # Instance type for node group
    desiredCapacity: 2 # The number of worker node in EKS Cluster
    volumeSize: 30  # EBS Volume for worker node (unit: GiB)
    volumeType: 'gp3'
    maxPodsPerNode: 100
    privateNetworking: false
    ssh:
      enableSsm: true
    iam:
      withAddonPolicies:
        imageBuilder: true # Add permission for Amazon ECR
        albIngress: true  # Add permission for ALB Ingress
        cloudWatch: true # Add permission for CloudWatch
        autoScaler: true # Add permission Auto Scaling
        ebs: true # Add permission EBS CSI driver
EOF

  • 创建集群stack

    1
    eksctl create cluster -f ${CLUSTER_NAME}-cluster.yaml

    整个过程大概耗时15分钟

  • OIDC

    1
    2
    3
    4
    5
    6
    aws eks describe-cluster --name ${CLUSTER_NAME} --query "cluster.identity.oidc.issuer" --output text

    eksctl utils associate-iam-oidc-provider \
        --region ${AWS_REGION} \
        --cluster ${CLUSTER_NAME} \
        --approve

  • 创建 IAM Policy

    1
    curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.5.4/docs/install/iam_policy.json
    1
    2
    3
    aws iam create-policy \
        --policy-name  AWSLoadBalancerControllerIAMPolicy_${CLUSTER_NAME}  \
        --policy-document file://iam_policy.json

安装helm

1
2
3
4
5
6
7
8
9
10
11
12
helm repo add eks https://aws.github.io/eks-charts

helm repo update

  helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
  -n kube-system \
  --set clusterName=${CLUSTER_NAME} \
  --set serviceAccount.create=false \
  --set serviceAccount.name=aws-load-balancer-controller \
  --set enableShield=false \
  --set enableWaf=false \
  --set enableWafv2=false
  • 验证安装
    1
    kubectl get deployment -n kube-system aws-load-balancer-controller

安装EBS CSI驱动

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
eksctl create iamserviceaccount    \
--name ebs-csi-controller-sa     \
--namespace kube-system     \
--cluster ${CLUSTER_NAME}     \
--role-name AmazonEKS_EBS_CSI_DriverRole_${CLUSTER_NAME}     \
--role-only     \
--attach-policy-arn arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy \
--approve


eksctl create addon \
--name aws-ebs-csi-driver \
--cluster ${CLUSTER_NAME} \
--service-account-role-arn arn:aws:iam::${ACCOUNT_ID}:role/AmazonEKS_EBS_CSI_DriverRole_${CLUSTER_NAME} \
–force

安装 KubeSphere

1
2
3
kubectl apply -f https://github.com/kubesphere/ks-installer/releases/download/v3.4.1/kubesphere-installer.yaml

kubectl apply -f https://github.com/kubesphere/ks-installer/releases/download/v3.4.1/cluster-configuration.yaml

  • 检查日志

    1
    kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l 'app in (ks-install, ks-installer)' -o jsonpath='{.items[0].metadata.name}') -f

  • 修改公网访问

    1
    kubectl edit svc ks-console -n kubesphere-system

    在 metadata.annotations下新增:

    1
    2
    3
    service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
    service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
    service.beta.kubernetes.io/aws-load-balancer-type: external

    并把:NodePort 修改为 LoadBalancer