Openstack云平台脚本部署之计算节点服务配置（十五）
目录
一、简介
计算节点上服务配置相对于控制节点和网络节点来说比较简单，很多都是配置Openstack服务客户端，这里主要进行集成Ceph的配置，并配置Openstack的nova、neutron、ceilometer的客户端，具体不再赘述，配置内容参考脚本及文档。
二、部署脚本
脚本部署比较简单，操作在控制节点进行，首先执行 install-configure-ceph-auth-client-key.sh进行集成Ceph操作（这个步骤会在控制节点和计算节点配置），完成认证配置，然后再执行脚本install-configure-compute-nodes-services.sh，完成Openstack计算节点上服务配置。
集成Ceph，添加Ceph认证授权，脚本 install-configure-ceph-auth-client-key.sh如下：
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 | #!/bin/sh . ../0-set-config.sh ./style/print-split.sh "Ceph Authentication Installation" ###复制ceph配置文件 glance-api, cinder-volume, nova-compute and cinder-backup的主机名,由于存储和计算在同一个节点，不需要复制到自身 ./pssh-exe C "mkdir -p /etc/ceph/" scp $compute_host:/etc/ceph/ceph.conf /etc/ceph/ceph.conf ./scp-exe C /etc/ceph/ceph.conf /etc/ceph/ceph.conf ###[所有控制节点]在glance-api节点上 ./pssh-exe C "yum install -y python-rbd" ###[所有控制节点]在nova-compute, cinder-backup 和cinder-volume节点上 ./pssh-exe C "yum install -y ceph-common" ###安装Ceph客户端认证[这里放在控制节点执行执行，] ceph auth get-or-create client.glance mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=images' ceph auth get-or-create client.cinder-backup mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=backups' ceph auth get-or-create client.cinder mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=volumes, allow rwx pool=vms, allow rwx pool=images' ####为client.cinder, client.glance, and client.cinder-backup添加keyring . style/print-info.sh "Copy cinder.keyring & glance.keyring & cinder-backup.keyring to compute nodes" for ((i=0; i<${#controller_map[@]}; i+=1)); do name=${controller_name[$i]}; ip=${controller_map[$name]}; ceph auth get-or-create client.glance | ssh $name tee /etc/ceph/ceph.client.glance.keyring ssh $name chown glance:glance /etc/ceph/ceph.client.glance.keyring ceph auth get-or-create client.cinder | ssh $name tee /etc/ceph/ceph.client.cinder.keyring ssh $name chown cinder:cinder /etc/ceph/ceph.client.cinder.keyring ceph auth get-or-create client.cinder-backup | ssh $name tee /etc/ceph/ceph.client.cinder-backup.keyring ssh $name chown cinder:cinder /etc/ceph/ceph.client.cinder-backup.keyring done; ###复制Keyring文件到nova-compute节点,为nova-compute节点上创建临时密钥 . style/print-info.sh "Copy cinder.keyring to compute nodes" for ((i=0; i<${#hypervisor_map[@]}; i+=1)); do name=${hypervisor_name[$i]}; ip=${hypervisor_map[$name]}; ceph auth get-or-create client.cinder | ssh $name tee /etc/ceph/ceph.client.cinder.keyring ceph auth get-key client.cinder | ssh $name tee client.cinder.key done; |
在每个计算节点安装配置Openstack服务，脚本install-configure-compute-nodes-services.sh如下：
| 1 2 3 4 5 6 | #!/bin/sh . ../0-set-config.sh ./style/print-split.sh "Openstack Services Installation on Compute Nodes" ./scp-exe H compute_nodes_exec.sh /tmp ./pssh-exe H "chmod +x /tmp/compute_nodes_exec.sh" ./pssh-exe H "/tmp/compute_nodes_exec.sh $virtual_ip $local_nic $data_nic $password" |
配置方式采用了远程执行配置脚本，首先会将compute_nodes_exec.sh拷贝到每个计算节点上，然后再远程执行的该脚本，compute_nodes_exec.sh如下：
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 | #!/bin/sh vip=$1 local_nic=$2 data_nic=$3 password=$4 yum install -y centos-release-openstack-mitaka yum install -y python-openstackclient openstack-selinux openstack-utils ### 安装组件 yum install -y openstack-nova-compute yum install -y openstack-neutron-openvswitch yum install -y openstack-neutron openstack-neutron-ml2 openstack-neutron-openvswitch yum install -y ceph-common yum install -y openstack-ceilometer-compute python-ceilometerclient python-pecan ### 1. OpenStack Compute service ### 修改配置文件/etc/nova/nova.conf openstack-config --set /etc/nova/nova.conf DEFAULT my_ip $(ip addr show dev $local_nic scope global | grep "inet " | sed -e 's#.*inet ##g' -e 's#/.*##g') openstack-config --set /etc/nova/nova.conf DEFAULT use_neutron True openstack-config --set /etc/nova/nova.conf DEFAULT firewall_driver nova.virt.firewall.NoopFirewallDriver openstack-config --set /etc/nova/nova.conf DEFAULT memcached_servers controller01:11211,controller02:11211,controller03:11211 openstack-config --set /etc/nova/nova.conf DEFAULT rpc_backend rabbit openstack-config --set /etc/nova/nova.conf oslo_messaging_rabbit rabbit_hosts controller01:5672,controller02:5672,controller03:5672 openstack-config --set /etc/nova/nova.conf oslo_messaging_rabbit rabbit_ha_queues true openstack-config --set /etc/nova/nova.conf oslo_messaging_rabbit rabbit_retry_interval 1 openstack-config --set /etc/nova/nova.conf oslo_messaging_rabbit rabbit_retry_backoff 2 openstack-config --set /etc/nova/nova.conf oslo_messaging_rabbit rabbit_max_retries 0 openstack-config --set /etc/nova/nova.conf oslo_messaging_rabbit rabbit_durable_queues true openstack-config --set /etc/nova/nova.conf oslo_messaging_rabbit rabbit_userid openstack openstack-config --set /etc/nova/nova.conf oslo_messaging_rabbit rabbit_password $password openstack-config --set /etc/nova/nova.conf DEFAULT auth_strategy keystone openstack-config --set /etc/nova/nova.conf keystone_authtoken auth_uri http://$vip:5000 openstack-config --set /etc/nova/nova.conf keystone_authtoken auth_url http://$vip:35357 openstack-config --set /etc/nova/nova.conf keystone_authtoken memcached_servers controller01:11211,controller02:11211,controller03:11211 openstack-config --set /etc/nova/nova.conf keystone_authtoken auth_type password openstack-config --set /etc/nova/nova.conf keystone_authtoken project_domain_name default openstack-config --set /etc/nova/nova.conf keystone_authtoken user_domain_name default openstack-config --set /etc/nova/nova.conf keystone_authtoken project_name service openstack-config --set /etc/nova/nova.conf keystone_authtoken username nova openstack-config --set /etc/nova/nova.conf keystone_authtoken password $password openstack-config --set /etc/nova/nova.conf oslo_concurrency lock_path /var/lib/nova/tmp openstack-config --set /etc/nova/nova.conf vnc enabled True openstack-config --set /etc/nova/nova.conf vnc vncserver_listen 0.0.0.0 openstack-config --set /etc/nova/nova.conf vnc vncserver_proxyclient_address $(ip addr show dev $local_nic scope global | grep "inet " | sed -e 's#.*inet ##g' -e 's#/.*##g') openstack-config --set /etc/nova/nova.conf vnc novncproxy_base_url http://$vip:6080/vnc_auto.html openstack-config --set /etc/nova/nova.conf glance api_servers http://$vip:9292 openstack-config --set /etc/nova/nova.conf libvirt virt_type $(count=$(egrep -c '(vmx|svm)' /proc/cpuinfo); if [ $count -eq 0 ];then echo "qemu"; else echo "kvm"; fi) ### 打开虚拟机迁移的监听端口 sed -i -e "s#\#listen_tls *= *0#listen_tls = 0#g" /etc/libvirt/libvirtd.conf sed -i -e "s#\#listen_tcp *= *1#listen_tcp = 1#g" /etc/libvirt/libvirtd.conf sed -i -e "s#\#auth_tcp *= *\"sasl\"#auth_tcp = \"none\"#g" /etc/libvirt/libvirtd.conf sed -i -e "s#\#LIBVIRTD_ARGS *= *\"--listen\"#LIBVIRTD_ARGS=\"--listen\"#g" /etc/sysconfig/libvirtd #### 2. OpenStack Network service ### 修改/etc/neutron/neutron.conf openstack-config --set /etc/neutron/neutron.conf DEFAULT rpc_backend rabbit openstack-config --set /etc/neutron/neutron.conf oslo_messaging_rabbit rabbit_hosts controller01:5672,controller02:5672,controller03:5672 openstack-config --set /etc/neutron/neutron.conf oslo_messaging_rabbit rabbit_ha_queues true openstack-config --set /etc/neutron/neutron.conf oslo_messaging_rabbit rabbit_retry_interval 1 openstack-config --set /etc/neutron/neutron.conf oslo_messaging_rabbit rabbit_retry_backoff 2 openstack-config --set /etc/neutron/neutron.conf oslo_messaging_rabbit rabbit_max_retries 0 openstack-config --set /etc/neutron/neutron.conf oslo_messaging_rabbit rabbit_durable_queues true openstack-config --set /etc/neutron/neutron.conf oslo_messaging_rabbit rabbit_userid openstack openstack-config --set /etc/neutron/neutron.conf oslo_messaging_rabbit rabbit_password $password openstack-config --set /etc/neutron/neutron.conf DEFAULT auth_strategy keystone openstack-config --set /etc/neutron/neutron.conf keystone_authtoken auth_uri http://$vip:5000 openstack-config --set /etc/neutron/neutron.conf keystone_authtoken auth_url http://$vip:35357 openstack-config --set /etc/neutron/neutron.conf keystone_authtoken memcached_servers controller01:11211,controller02:11211,controller03:11211 openstack-config --set /etc/neutron/neutron.conf keystone_authtoken auth_type password openstack-config --set /etc/neutron/neutron.conf keystone_authtoken project_domain_name default openstack-config --set /etc/neutron/neutron.conf keystone_authtoken user_domain_name default openstack-config --set /etc/neutron/neutron.conf keystone_authtoken project_name service openstack-config --set /etc/neutron/neutron.conf keystone_authtoken username neutron openstack-config --set /etc/neutron/neutron.conf keystone_authtoken password $password openstack-config --set /etc/neutron/neutron.conf oslo_concurrency lock_path /var/lib/neutron/tmp ### 配置Open vSwitch agent，/etc/neutron/plugins/ml2/openvswitch_agent.ini，注意，此处填写第二块网卡 openstack-config --set /etc/neutron/plugins/ml2/openvswitch_agent.ini securitygroup enable_security_group True openstack-config --set /etc/neutron/plugins/ml2/openvswitch_agent.ini securitygroup enable_ipset True openstack-config --set /etc/neutron/plugins/ml2/openvswitch_agent.ini securitygroup firewall_driver iptables_hybrid openstack-config --set /etc/neutron/plugins/ml2/openvswitch_agent.ini ovs local_ip $(ip addr show dev $data_nic scope global | grep "inet " | sed -e 's#.*inet ##g' -e 's#/.*##g') openstack-config --set /etc/neutron/plugins/ml2/openvswitch_agent.ini agent tunnel_types vxlan openstack-config --set /etc/neutron/plugins/ml2/openvswitch_agent.ini agent l2_population False ### 配置nova和neutron集成，/etc/nova/nova.conf openstack-config --set /etc/nova/nova.conf neutron url http://$vip:9696 openstack-config --set /etc/nova/nova.conf neutron auth_url http://$vip:35357 openstack-config --set /etc/nova/nova.conf neutron auth_type password openstack-config --set /etc/nova/nova.conf neutron project_domain_name default openstack-config --set /etc/nova/nova.conf neutron user_domain_name default openstack-config --set /etc/nova/nova.conf neutron region_name RegionOne openstack-config --set /etc/nova/nova.conf neutron project_name service openstack-config --set /etc/nova/nova.conf neutron username neutron openstack-config --set /etc/nova/nova.conf neutron password $password ln -s /etc/neutron/plugins/ml2/ml2_conf.ini /etc/neutron/plugin.ini ### 3. OpenStack Block Storage service ###计算节点安装客户端命令行工具 echo "<secret ephemeral='no' private='no'> <uuid>032198f4-b815-4254-9de2-185f935bd7de</uuid> <usage type='ceph'> <name>client.cinder secret</name> </usage> </secret>">secret.xml virsh secret-define --file secret.xml virsh secret-set-value --secret 032198f4-b815-4254-9de2-185f935bd7de --base64 $(cat /etc/ceph/ceph.client.cinder.keyring |grep 'key ='|awk '{print $3}') && rm secret.xml ###设置/etc/nova/nova.conf openstack-config --set /etc/nova/nova.conf libvirt images_type rbd openstack-config --set /etc/nova/nova.conf libvirt images_rbd_pool vms openstack-config --set /etc/nova/nova.conf libvirt images_rbd_ceph_conf /etc/ceph/ceph.conf openstack-config --set /etc/nova/nova.conf libvirt rbd_user cinder openstack-config --set /etc/nova/nova.conf libvirt rbd_secret_uuid $(virsh secret-list| grep ceph| awk '{print $1}') openstack-config --set /etc/nova/nova.conf libvirt disk_cachemodes \"network=writeback\" openstack-config --set /etc/nova/nova.conf libvirt inject_password false openstack-config --set /etc/nova/nova.conf libvirt inject_key false openstack-config --set /etc/nova/nova.conf libvirt inject_partition -2 openstack-config --set /etc/nova/nova.conf libvirt live_migration_flag "VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE,VIR_MIGRATE_PERSIST_DEST,VIR_MIGRATE_TUNNELLED" ### 4. OpenStack Ceilomerter service ###[在计算节点上配置] openstack-config --set /etc/ceilometer/ceilometer.conf DEFAULT rpc_backend rabbit openstack-config --set /etc/ceilometer/ceilometer.conf oslo_messaging_rabbit rabbit_hosts controller01:5672,controller02:5672,controller03:5672 openstack-config --set /etc/ceilometer/ceilometer.conf oslo_messaging_rabbit rabbit_ha_queues true openstack-config --set /etc/ceilometer/ceilometer.conf oslo_messaging_rabbit rabbit_retry_interval 1 openstack-config --set /etc/ceilometer/ceilometer.conf oslo_messaging_rabbit rabbit_retry_backoff 2 openstack-config --set /etc/ceilometer/ceilometer.conf oslo_messaging_rabbit rabbit_max_retries 0 openstack-config --set /etc/ceilometer/ceilometer.conf oslo_messaging_rabbit rabbit_durable_queues true openstack-config --set /etc/ceilometer/ceilometer.conf oslo_messaging_rabbit rabbit_userid openstack openstack-config --set /etc/ceilometer/ceilometer.conf oslo_messaging_rabbit rabbit_password $password openstack-config --set /etc/ceilometer/ceilometer.conf DEFAULT auth_strategy keystone openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken auth_uri http://$vip:5000 openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken auth_url http://$vip:35357 openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken memcached_servers controller01:11211,controller02:11211,controller03:11211 openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken auth_type password openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken project_domain_name default openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken user_domain_name default openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken project_name service openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken username ceilometer openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken password $password openstack-config --set /etc/ceilometer/ceilometer.conf service_credentials auth_type password openstack-config --set /etc/ceilometer/ceilometer.conf service_credentials auth_url http://$vip:5000/v3 openstack-config --set /etc/ceilometer/ceilometer.conf service_credentials project_domain_name default openstack-config --set /etc/ceilometer/ceilometer.conf service_credentials user_domain_name default openstack-config --set /etc/ceilometer/ceilometer.conf service_credentials project_name service openstack-config --set /etc/ceilometer/ceilometer.conf service_credentials username ceilometer openstack-config --set /etc/ceilometer/ceilometer.conf service_credentials password $password openstack-config --set /etc/ceilometer/ceilometer.conf service_credentials interface internalURL openstack-config --set /etc/ceilometer/ceilometer.conf service_credentials region_name RegionOne ### 配置nova使用ceilometer服务 openstack-config --set /etc/nova/nova.conf DEFAULT instance_usage_audit True openstack-config --set /etc/nova/nova.conf DEFAULT instance_usage_audit_period hour openstack-config --set /etc/nova/nova.conf DEFAULT notify_on_state_change vm_and_task_state openstack-config --set /etc/nova/nova.conf DEFAULT notification_driver messagingv2 ###启动服务 systemctl enable libvirtd.service openstack-nova-compute.service systemctl start libvirtd.service openstack-nova-compute.service systemctl enable openstack-nova-compute.service systemctl restart openstack-nova-compute.service systemctl start openvswitch.service systemctl restart neutron-openvswitch-agent.service systemctl enable neutron-openvswitch-agent.service ###启动服务 systemctl enable openstack-ceilometer-compute.service systemctl start openstack-ceilometer-compute.service systemctl restart openstack-nova-compute.service |
三、参考文档
OpenStack Docs: Install and configure a compute node
OpenStack Docs: Install and configure compute node
Configuring the compute node — openstackhaguide 0.0.1.dev824 documentation
OpenStack Docs: Enable Compute service meters
Block Devices and OpenStack — Ceph Documentation
OpenStack Docs: Scenario: Legacy with Open vSwitch
四、源码
脚本源码：https://github.com/zjmeixinyanzhi/Openstack-HA-Install-Shells
五、系列文章
“Openstack云平台脚本部署”系列文章目录如下：
Openstack云平台脚本部署之Galera高可用集群配置（二）
Openstack云平台脚本部署之RabbitMQ高可用集群部署（三）
Openstack云平台脚本部署之Memcached配置（五）
Openstack云平台脚本部署之Keystone认证服务配置（六）
Openstack云平台脚本部署之Glance镜像服务配置（七）
Openstack云平台脚本部署之Nova计算服务配置（八）
Openstack云平台脚本部署之Neutron网络服务配置（九）
Openstack云平台脚本部署之Dashboard配置（十）
Openstack云平台脚本部署之Cinder块存储服务配置（十一）
Openstack云平台脚本部署之Ceilometer数据收集服务配置（十二）
Openstack云平台脚本部署之Aodh告警服务配置（十三）
Openstack云平台脚本部署之Ceph存储集群配置（十四）