基于kubernetes构建Docker集群-安装篇

1,环境说明

组件版本说明：

系统版本：ceontos 7

kubernetes版本：0.17.1

etcd版本：2.1.1

docker版本：1.6.2

环境说明

etcd：172.16.0.3

master：172.16.0.2 kubernetes+docker

minion1：172.16.0.4 kubernetes+docker

minion2：172.16.0.5 kubernetes+docker

 

2,系统环境配置

更新yum源

# yum -y install wget ntpdate bind-utils

# wget http://mirror.centos.org/centos/7/extras/x86_64/Packages/epel-release-7-2.noarch.rpm

# yum update

 

防火墙设置（根据个人情况配置，非必须）

关闭firewall

# systemctl stop firewalld.service #停止firewall

# systemctl disable firewalld.service #禁止firewall开机启动

 

安装iptalbles

# yum install iptables-services #安装

# systemctl start iptables.service #最后重启防火墙使配置生效

# systemctl enable iptables.service #设置防火墙开机启动

 

2,安装配置etcd

2.1，安装

# yum install etcd

2.2，配置

[root@etcd ~]# grep -Ev "^#|^$" /etc/etcd/etcd.conf

ETCD_NAME=default

ETCD_DATA_DIR="/var/lib/etcd/default.etcd"

ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:4001"

ETCD_ADVERTISE_CLIENT_URLS="http://0.0.0.0:4001"

2.3,启动

[root@etcd ~]# systemctl start etcd.service

2.4,验证

[root@etcd ~]# etcd -version

etcd Version: 2.0.13

Git SHA: 92e3895

Go Version: go1.4.2

Go OS/Arch: linux/amd64

 

#在master上

[root@master ~]# telnet 172.16.0.3 4001

 

 

2.5,配置flannel

[root@etcd ~]# etcdctl mk /coreos.com/network/config \'{"Network":"172.17.0.0/16"}\'

{"Network":"172.17.0.0/16"}

[root@etcd ~]# etcdctl get /coreos.com/network/config

{"Network":"172.17.0.0/16"}

 

 

3,安装k8s

服务器：所有服务器

#yum install kubernetes

 

升级方法：

# mkdir -p /home/install && cd /home/install

# wget https://github.com/GoogleCloudPlatform/kubernetes/releases/download/v0.6.2/kubernetes.tar.gz

# tar -zxvf kubernetes.tar.gz

# tar -zxvf kubernetes/server/kubernetes-server-linux-amd64.tar.gz

# cp kubernetes/server/bin/kube* /usr/bin

 

3.1,master配置k8s

master运行三个组件,包括apiserver、scheduler、controller-manager，相关配置项也只涉及这三块。

[/etc/kubernetes/config]

[root@master ~]# grep -Ev "^$|^#" /etc/kubernetes/config

KUBE_LOGTOSTDERR="--logtostderr=true"

KUBE_LOG_LEVEL="--v=0"

KUBE_ALLOW_PRIV="--allow_privileged=false"

KUBE_MASTER="--master=http:://172.16.0.2:8080"

 

[/etc/kubernetes/apiserver]

[root@master ~]# grep -Ev "^$|^#" /etc/kubernetes/apiserver

KUBE_API_ADDRESS="--address=0.0.0.0"

KUBE_API_PORT="--port=8080"

KUBELET_PORT="--kubelet_port=10250"

KUBE_ETCD_SERVERS="--etcd_servers=http://127.0.0.1:4001"

KUBE_SERVICE_ADDRESSES="--portal_net=10.254.0.0/16"

KUBE_ADMISSION_CONTROL="--admission_control=NamespaceAutoProvision,LimitRanger,ResourceQuota"

KUBE_API_ARGS=""

 

[/etc/kubernetes/controller-manager]

[root@master ~]# grep -Ev "^$|^#" /etc/kubernetes/controller-manager

KUBELET_ADDRESSES="--machines=127.0.0.1,172.16.0.4,172.16.0.5"

KUBE_CONTROLLER_MANAGER_ARGS=""

 

[/etc/kubernetes/scheduler]

[root@master ~]# grep -Ev "^$|^#" /etc/kubernetes/scheduler

KUBE_SCHEDULER_ARGS=""

 

 

3.2,master上启动k8s服务

# systemctl start kube-apiserver.service kube-controller-manager.service kube-scheduler.service

# systemctl enable kube-apiserver.service kube-controller-manager.service kube-scheduler.service

 

3.3,查看k8s版本

[root@master ~]# kubectl version

Client Version: version.Info{Major:"1", Minor:"0+", GitVersion:"v1.0.0-290-gb2dafdaef5acea", GitCommit:"b2dafdaef5aceafad503ab56254b60f80da9e980", GitTreeState:"clean"}

Server Version: version.Info{Major:"1", Minor:"0+", GitVersion:"v1.0.0-290-gb2dafdaef5acea", GitCommit:"b2dafdaef5aceafad503ab56254b60f80da9e980", GitTreeState:"clean"}

 

报错：

[root@master ~]# kubectl version

Client Version: version.Info{Major:"1", Minor:"0+", GitVersion:"v1.0.0-290-gb2dafdaef5acea", GitCommit:"b2dafdaef5aceafad503ab56254b60f80da9e980", GitTreeState:"clean"}

error: couldn\'t read version from server: Get http://localhost:8080/api: dial tcp 127.0.0.1:8080: connection refused

 

解决：

需要配置k8s，见上！

 

 

 

3.4,minion配置k8s

minion运行两个组件，kubelet proxy，对应的配置是config和kubelet

minion上还需要配置docker，见3.5

 

[/etc/kubernetes/config]

[root@minion1 ~]# grep -Ev "^$|^#" /etc/kubernetes/config

KUBE_LOGTOSTDERR="--logtostderr=true"

KUBE_LOG_LEVEL="--v=0"

KUBE_ALLOW_PRIV="--allow_privileged=false"

KUBE_MASTER="--master=http://172.16.0.2:8080"

 

 

[root@localhost ~]# grep -Ev "^$|^#" /etc/kubernetes/kubelet

KUBELET_ADDRESS="--address=0.0.0.0"

KUBELET_HOSTNAME="--hostname_override=172.16.0.4"

KUBELET_API_SERVER="--api_servers=http://172.16.0.2:8080"

KUBELET_ARGS=""

 

3.5,minion上配置docker

配置docker，以便远程管理。

[root@minion1 ~]# grep -Ev "^$|^#" /etc/sysconfig/docker

OPTIONS=\'--selinux-enabled -H tcp://0.0.0.0:2375 -H fd://\'

DOCKER_CERT_PATH=/etc/docker

 

#启动docker的时候可能会报错，可以先不配置

 

3.6,配置flanneld

[root@minion1 ~]# grep -Ev "^$|^#" /etc/sysconfig/flanneld

FLANNEL_ETCD="http://172.16.0.3:4001"

FLANNEL_ETCD_KEY="/coreos.com/network"

 

3.7,minion上启动k8s，docker，flanneld

[root@minion1 ~]# systemctl start docker.service flanneld.service

[root@minion1 ~]# systemctl start kubelet.service kube-proxy.service

 

如果出现docker0和flannel设置的ip地址不同，则可采取如下方式修改

#systemctl stop docker

#ifconfig docker0 down

#brctl delbr docker0

#systemctl start docker

 

3.8,daocker启动报错

[root@localhost sysconfig]# systemctl start docker

Job for docker.service failed. See \'systemctl status docker.service\' and \'journalctl -xn\' for details.

[root@localhost sysconfig]# systemctl status docker.service

docker.service - Docker Application Container Engine

Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled)

Drop-In: /usr/lib/systemd/system/docker.service.d

└─flannel.conf

Active: failed (Result: exit-code) since 三 2015-09-16 14:18:47 CST; 11s ago

Docs: http://docs.docker.com

Process: 9150 ExecStart=/usr/bin/docker -d $OPTIONS $DOCKER_STORAGE_OPTIONS $DOCKER_NETWORK_OPTIONS $ADD_REGISTRY $BLOCK_REGISTRY $INSECURE_REGISTRY (code=exited, status=1/FAILURE)

Main PID: 9150 (code=exited, status=1/FAILURE)

 

9月 16 14:18:47 localhost.localdomain systemd[1]: Starting Docker Application Container Engine...

9月 16 14:18:47 localhost.localdomain docker[9150]: time="2015-09-16T14:18:47.842291856+08:00" level=info msg="Listening for...sock)"

9月 16 14:18:47 localhost.localdomain docker[9150]: time="2015-09-16T14:18:47.861153138+08:00" level=error msg="WARNING: No ...n use"

9月 16 14:18:47 localhost.localdomain docker[9150]: time="2015-09-16T14:18:47.889459632+08:00" level=info msg="[graphdriver]...per""

9月 16 14:18:47 localhost.localdomain docker[9150]: time="2015-09-16T14:18:47.902509183+08:00" level=warning msg="Running mo...tus 1"

9月 16 14:18:47 localhost.localdomain docker[9150]: time="2015-09-16T14:18:47.907255506+08:00" level=info msg="Firewalld run...false"

9月 16 14:18:47 localhost.localdomain docker[9150]: time="2015-09-16T14:18:47.949811560+08:00" level=fatal msg="Error starti....61.1"

9月 16 14:18:47 localhost.localdomain systemd[1]: docker.service: main process exited, code=exited, status=1/FAILURE

9月 16 14:18:47 localhost.localdomain systemd[1]: Failed to start Docker Application Container Engine.

9月 16 14:18:47 localhost.localdomain systemd[1]: Unit docker.service entered failed state.

Hint: Some lines were ellipsized, use -l to show in full.

[root@localhost sysconfig]# docker -d

INFO[0000] Listening for HTTP on unix (/var/run/docker.sock)

ERRO[0000] WARNING: No --storage-opt dm.thinpooldev specified, using loopback; this configuration is strongly discouraged for production use

INFO[0000] [graphdriver] using prior storage driver "devicemapper"

WARN[0000] Running modprobe bridge nf_nat br_netfilter failed with message: , error: exit status 1

INFO[0000] Firewalld running: false

INFO[0000] Loading containers: start.

 

INFO[0000] Loading containers: done.

INFO[0000] Daemon has completed initialization

INFO[0000] Docker daemon commit=3043001/1.7.1 execdriver=native-0.2 graphdriver=devicemapper version=1.7.1

 

解决方法：

如果出现docker0和flannel设置的ip地址不同，则可采取如下方式修改

#systemctl stop docker

#ifconfig docker0 down

#brctl delbr docker0

#systemctl start docker

 

注意：在用虚拟机做测试的时候，假如克隆了minion的虚拟机，两台ninion虚拟机的docker0和flannel网段地址会一样，会造成NotReady状态。所以需要克隆master虚拟主机，配置成minion状态。

 

 

4,集群操作

查看node信息

[root@master ~]# kubectl get nodes

报错1：

Error from server: 501: All the given peers are not reachable (failed to propose on members [http://172.16.0.3:4001] twice [last error: Get http://172.16.0.3:4001/v2/keys/registry/minions?quorum=false&recursive=true&sorted=true: dial tcp 172.16.0.3:4001: i/o timeout]) [0]

原因：

docker没有注册到etcd上，查看docker未启动。

 

报错2：

[root@localhost ~]# kubectl get nodes

NAME LABELS STATUS

127.0.0.1 kubernetes.io/hostname=127.0.0.1 NotReady

172.16.0.2 kubernetes.io/hostname=172.16.0.2 NotReady

172.16.0.4 kubernetes.io/hostname=172.16.0.4 NotReady

172.16.0.5 kubernetes.io/hostname=172.16.0.5 NotReady

 

原因：

minion注册etcd有问题，需要查看etcd服务的端口是否是2379（etcd Version: 2.0.13

），在minion上是否能telnet通2379端口