DevOps/Kubernetes
📚[Sprint1] 쿠버네티스 무게감 있게 설치하기
S.A.H
2023. 9. 21. 23:35
반응형
*해당 글은 인프런 『쿠버네티스 어나더 클래스 (지상편) - Sprint1』 강의를 기반으로 복습&정리 차 작성되었습니다.
📌실습 환경 구성
이번에는 쿠버네티스 실습환경을 구성해보고자 합니다.
개인적으로 공부할때는 VMware관련 제품을 사용하다보니 VMware Workstation을 통해서 진행하려고 했으나, 첫 실습이기 때문에 실습내용에 더 집중하고자 VirtualBox로 진행했습니다.
🔍Vagrant를 활용하여 VM 생성
* OS: Rocky Linux
* 실습 스크립트를 활용하여 k8s 마스터 노드 생성
# Vagrant 폴더 생성
D:\> mkdir k8s
# Vagrant 스크립트 다운로드
D:\k8s> curl -O https://raw.githubusercontent.com/k8s-1pro/install/main/ground/k8s-1.27/vagrant-2.3.4/Vagrantfile
# Rocky Linux Repo 세팅
D:\k8s> curl -O https://raw.githubusercontent.com/k8s-1pro/install/main/ground/k8s-1.27/vagrant-2.3.4/rockylinux-repo.json
D:\k8s> vagrant box add rockylinux-repo.json
# Vagrant Disk 설정 Plugin 설치
D:\k8s> vagrant plugin install vagrant-vbguest vagrant-disksize
# Vagrant 실행 (VM생성)
D:\k8s> vagrant up스크립트를 이용하면 단 몇 분만에 마스터 노드 생성이 가능해집니다.. 여기서 그치면 안되겠죠..?
어떻게 스크립트가 구성되어 마스터 노드가 생성되었는지 아래와 같은 순서에 따라 그 과정을 천천히 살펴보면 이해하는 데 도움이 됩니다.
📌스크립트 맛보고 뜯어보기
🔍OS 구성
Vagrant.configure("2") do |config|
config.vm.box = "rockylinux/8" → OS 종류 선택
config.disksize.size = "50GB" → 디스크 크기 설정
config.vbguest.installer_options = { allow_kernel_upgrade: true }
config.vbguest.auto_update = false
config.vm.provision :shell, privileged: true, inline: $install_default
config.vm.define "master-node" do |master| → VM Name 설정
master.vm.hostname = "k8s-master" → hostname 설정
master.vm.network "private_network", ip: "192.168.56.30" → ip 설정(private 대역)
master.vm.provider :virtualbox do |vb|
vb.memory = 6144 → 메모리 설정
vb.cpus = 4 → CPU 설정
vb.customize ["modifyvm", :id, "--firmware", "efi"]
end
master.vm.provision :shell, privileged: true, inline: $install_master
end
end
$install_default = <<-SHELL → node별 기본 설치 언어 설정
네트워크는 Private-Network로 내부 통신을 하고, NAT를 통해 외부 인터넷망과 통신하여 Kubernetes설치 시 필요한 패키지들을 받을 수 있도록 하였습니다.
🔍 컨테이너 런타임 및 쿠버네티스 설치
기본적인 타임존 설정과 패키지 업데이트를 진행 한 후 본격적으로 쿠버네티스를 설치하기 위한 작업을 시작합니다.
# 방화벽 off 및 비활성화
[root@k8s-master ~]# systemctl stop firewalld
[root@k8s-master ~]# systemctl disable firewalld# swap 비활성화
[root@k8s-master ~]# swapoff -a && sed -i '/ swap / s/^/#/' /etc/fstab
# swap 사용률 0 확인
[root@k8s-master ~]# free
total used free shared buff/cache available
Mem: 6063280 1599260 130720 19168 4333300 4161612
Swap: 0 0 0
# fstab에 swap 부분 주석처리 확인
[root@k8s-master ~]# cat /etc/fstab | grep swap
#/swapfile none swap defaults 0 0
IPv4를 포워딩하여 iptables가 브리지된 트래픽을 보게 하기
(리눅스 노드의 iptables가 브리지된 트래픽을 올바르게 보기 위한 요구 사항으로 해당 세팅 값을 확인)
# 설정 세팅 확인
[root@k8s-master ~]# cat /etc/modules-load.d/k8s.conf
overlay
br_netfilter
[root@k8s-master ~]# cat /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
# 모듈 적재 확인
[root@k8s-master ~]# lsmod | grep overlay
overlay 139264 35
[root@k8s-master ~]# lsmod | grep br_netfilter
br_netfilter 24576 0
bridge 290816 1 br_netfilter컨테이너 런타임 설치
# docker repo 설정
[root@k8s-master ~]# yum install -y yum-utils
[root@k8s-master ~]# yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
# containerd 설치
[root@k8s-master ~]# yum install -y containerd.io-1.6.21-3.1.el8
[root@k8s-master ~]# systemctl daemon-reload
[root@k8s-master ~]# systemctl enable --now containerd
# containerd CRI 활성화
[root@k8s-master ~]# sed -i 's/^disabled_plugins/#disabled_plugins/' /etc/containerd/config.toml
# ["cri"] 주석처리 확인
[root@k8s-master ~]# cat /etc/containerd/config.toml
# Copyright 2018-2022 Docker Inc.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# http://www.apache.org/licenses/LICENSE-2.0
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#disabled_plugins = ["cri"]
...
[root@k8s-master ~]# systemctl restart containerdkubeadm 설치
# kubernetes.repo 설정
cat /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
exclude=kubelet kubeadm kubectl
# repo 확인
[root@k8s-master yum.repos.d]# yum repolist enabled
repo id repo name
appstream Rocky Linux 8 - AppStream
baseos Rocky Linux 8 - BaseOS
docker-ce-stable Docker CE Stable - x86_64
extras Rocky Linux 8 - Extras
kubernetes Kubernetes
# SELinux 설정 확인
[root@k8s-master ~]# cat /etc/selinux/config
SELINUX=permissive → permissive 확인
[root@k8s-master ~]# sestatus
Current mode: permissive → permissive 확인
Mode from config file: permissive → permissive 확인
# kubelet, kubeadm, kubectl 패키지 설치
[root@k8s-master ~]# yum install -y kubelet-1.27.1-0.x86_64 kubeadm-1.27.1-0.x86_64 kubectl-1.27.1-0.x86_64 --disableexcludes=kubernetes
systemctl enable --now kubelet
🔍 마스터 노드 세팅
kubeadm으로 클러스터 생성
# pod 네트워크 세팅
[root@k8s-master ~]# kubeadm init --pod-network-cidr=20.96.0.0/12 --apiserver-advertise-address 192.168.56.30
# master node 상태확인
[root@k8s-master ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
k8s-master Ready control-plane 85m v1.27.1
# pod network cidr 설정 확인
[root@k8s-master ~]# kubectl cluster-info dump | grep -m 1 cluster-cidr
"--cluster-cidr=20.96.0.0/12",
# apiserver advertise address 적용 확인
[root@k8s-master ~]# kubectl cluster-info
Kubernetes control plane is running at https://192.168.56.30:6443
# kubernetes component pod 확인 (Running 확인)
[root@k8s-master ~]# kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-5d78c9869d-l8kpn 1/1 Running 0 85m
coredns-5d78c9869d-qftxg 1/1 Running 0 85m
etcd-k8s-master 1/1 Running 0 86m
kube-apiserver-k8s-master 1/1 Running 0 86m
kube-controller-manager-k8s-master 1/1 Running 0 86m
kube-proxy-tht2n 1/1 Running 0 85m
kube-scheduler-k8s-master 1/1 Running 0 86m
metrics-server-7db4fb59f9-r8zwv 1/1 Running 0 85m
# kubectl 사용 설정
[root@k8s-master ~]# mkdir -p $HOME/.kube
[root@k8s-master ~]# cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@k8s-master ~]# chown $(id -u):$(id -g) $HOME/.kube/config
# 인증서 설정 확인
[root@k8s-master ~]# cat ~/.kube/config
...
server: https://192.168.56.30:6443 → IP 설정 확인
...
# Pod Network 설치 (calico)
[root@k8s-master ~]# kubectl create -f https://raw.githubusercontent.com/k8s-1pro/install/main/ground/k8s-1.27/calico-3.25.1/calico.yaml
[root@k8s-master ~]# kubectl create -f https://raw.githubusercontent.com/k8s-1pro/install/main/ground/k8s-1.27/calico-3.25.1/calico-custom.yaml
# Calico Pod 상태 확인 (Running 확인)
[root@k8s-master ~]# kubectl get -n calico-system pod
NAME READY STATUS RESTARTS AGE
calico-kube-controllers-789dc4c76b-ls888 1/1 Running 0 97m
calico-node-qck7p 1/1 Running 0 97m
calico-typha-86d895795d-wghjr 1/1 Running 0 97m
csi-node-driver-69qsk 2/2 Running 0 97m
[root@k8s-master ~]# kubectl get -n calico-apiserver pod
NAME READY STATUS RESTARTS AGE
calico-apiserver-5c57b4569f-dd6qk 1/1 Running 0 96m
calico-apiserver-5c57b4569f-s5h6p 1/1 Running 0 96m
# Calico에 pod network cidr 적용 확인
[root@k8s-master ~]# kubectl get installations.operator.tigera.io default -o yaml | grep cidr
cidr: 20.96.0.0/12
cidr: 20.96.0.0/12
# Master에 Pod를 생성 할수 있도록 설정
[root@k8s-master ~]# kubectl taint nodes k8s-master node-role.kubernetes.io/control-plane-
# Taints 해제 확인
[root@k8s-master ~]# kubectl describe nodes | grep Taints
Taints: <none>
쿠버네티스 편의 기능 설치
# kubectl 자동완성 기능 설정
echo "source <(kubectl completion bash)" >> ~/.bashrc
echo 'alias k=kubectl' >>~/.bashrc
echo 'complete -o default -F __start_kubectl k' >>~/.bashrc
# kubectl 자동완성 기능 설정 확인
[root@k8s-master ~]# cat ~/.bashrc
# .bashrc
# User specific aliases and functions
alias rm='rm -i'
alias cp='cp -i'
alias mv='mv -i'
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
source <(kubectl completion bash)
alias k=kubectl
complete -o default -F __start_kubectl k
# Dashboard 설치
kubectl create -f https://raw.githubusercontent.com/k8s-1pro/install/main/ground/k8s-1.27/dashboard-2.7.0/dashboard.yaml
# Dashboard Running 확인
[root@k8s-master ~]# kubectl get pod -n kubernetes-dashboard
NAME READY STATUS RESTARTS AGE
dashboard-metrics-scraper-5cb4f4bb9c-rpwk4 1/1 Running 0 111m
kubernetes-dashboard-6bc7c98694-5d9bz 1/1 Running 0 111m
# Metrics Server 설치
kubectl create -f https://raw.githubusercontent.com/k8s-1pro/install/main/ground/k8s-1.27/metrics-server-0.6.3/metrics-server.yaml
# Metrics Server 설치 확인
[root@k8s-master ~]# kubectl get pod -n kube-system | grep metrics
metrics-server-7db4fb59f9-r8zwv 1/1 Running 0 112m🔍pod 상태 확인
[root@k8s-master ~]# k get pods -A
NAMESPACE NAME READY STATUS RESTARTS AGE
calico-apiserver calico-apiserver-6d7f77dc86-24nkn 1/1 Running 0 19m
calico-apiserver calico-apiserver-6d7f77dc86-9mjt6 1/1 Running 0 19m
calico-system calico-kube-controllers-789dc4c76b-zf2z6 1/1 Running 0 21m
calico-system calico-node-bsgs5 1/1 Running 0 21m
calico-system calico-typha-956d8d9f5-kwjq5 1/1 Running 0 21m
calico-system csi-node-driver-rwf22 2/2 Running 0 21m
kube-system coredns-5d78c9869d-j4dk6 1/1 Running 0 21m
kube-system coredns-5d78c9869d-zwwhn 1/1 Running 0 21m
kube-system etcd-k8s-master 1/1 Running 0 21m
kube-system kube-apiserver-k8s-master 1/1 Running 0 21m
kube-system kube-controller-manager-k8s-master 1/1 Running 0 21m
kube-system kube-proxy-gx62q 1/1 Running 0 21m
kube-system kube-scheduler-k8s-master 1/1 Running 0 21m
kube-system metrics-server-7db4fb59f9-jl8mz 1/1 Running 0 21m
kubernetes-dashboard dashboard-metrics-scraper-5cb4f4bb9c-2cr78 1/1 Running 0 21m
kubernetes-dashboard kubernetes-dashboard-6bc7c98694-zd7xz 1/1 Running 0 21m
tigera-operator tigera-operator-549d4f9bdb-svv47 1/1 Running 0 21m🔍 Kubernetes Dashboard 접속 및 상태 확인
https://192.168.56.30:30000/#/login
반응형