如何在OpenStack-Ansible上集成Tungsten Fabric(上)



  • 本文作者James Denton,拥有超过15年信息技术领域行业经验,目前为知名云计算服务商Rackspace Hosting首席架构师,致力于云网络和对基于OpenStack的Rackspace私有云产品的支持。

    Tungsten Fabric(前身为OpenContrail)是Linux基金会旗下的“多云、多技术栈”SDN解决方案。简而言之,Tungsten Fabric和Contrail(基于TF的商业产品)可以替代和增强标准OpenStack云的许多网络组件,并提供以下功能:
    分布式虚拟路由
    DHCP和元数据服务
    基于策略的访问控制
    与安全组的兼容性
    ……等等

    转发平面支持MPLS over GRE、VXLAN、L2/L3单播和L3多播,以实现虚拟网络和物理网络之间的互连。
    注:关于Tungsten Fabric架构的概述可在下面的链接找到:https://tungstenfabric.org.cn/docs

    我最近接受了将Tungsten Fabric集成到OpenStack-Ansible中的挑战,以简化TF和相关OpenStack bits在生产级OpenStack云中的部署。本文将在较高级别涵盖OpenStack-Ansible的主(Rocky)分支的一些补丁,以及Juniper和TF社区提供的用于部署Tungsten Fabric的contrail-ansible-deployer playbooks的一些调整。本文所描述的过程绝不意味着是最终的过程,可能会显得比较笨拙且不理想,但是,这是一个开始。

    注:本文作者指明文章中的Tungsten Fabric、OpenContrail和Contrail为混用,翻译时统一为Tungsten Fabric。

    集成要求

    几周前,我部署了一个独立的3节点Tungsten Fabric设置,其中包含基于Queens的OpenStack-Ansible的All-in-One节点。在弄清楚使事情进入半工作状态所需的调整后,我决定亲自尝试并部署一个AIO节点,该节点包含一个单一的Tungsten Fabric服务实例以及基本的OpenStack服务。

    以下为最低规格的配置建议:
    系统: Ubuntu VM
    OS: 16.04.4 LTS
    内存: 48GB
    硬盘: 300GB
    NIC: 单口网卡

    如我稍后指出的,裸机节点可能复杂度较小,但是ESXi或其它管理程序上的虚拟机应该是可以尝试的。

    从OpenStack-Ansible开始

    首先,请克隆OpenStack-Ansible存储库。在撰写本文时,master分支是与OpenStack的第18个发行版Rocky相关联的。

    # git clone https://git.openstack.org/openstack/openstack-ansible /opt/openstack-ansible
    # cd /opt/openstack-ansible
    # git checkout master
    # export ANSIBLE_ROLE_FETCH_MODE=git-clone
    

    接下来,运行引导程序脚本:

    # scripts/bootstrap-ansible.sh
    # scripts/bootstrap-aio.sh
    

    引导程序脚本将下载playbooks以部署OpenStack,还将在服务器上准备符合OpenStack-Ansible架构的网络环境。

    角色修改

    对使用OpenStack-Ansible部署的OpenStack云进行更改,通常意味着对构成部署的Ansible角色进行更改。这包括对任务、模板、变量等的更改。

    需要修改的角色包括:
    os_neutron
    os_nova

    是否所有这些角色的更改都是必需的,还有待观察,这里只是为了更好地说明。

    os_neutron

    一些新文件包括:

    root@aio1:/etc/ansible/roles/os_neutron# git diff --staged
    diff --git a/tasks/providers/opencontrail_config.yml b/tasks/providers/opencontrail_config.yml
    new file mode 100644
    index 0000000..8f5fc7d
    --- /dev/null
    +++ b/tasks/providers/opencontrail_config.yml
    @@ -0,0 +1,99 @@
    +---
    +# Copyright 2018, Rackspace Hosting, Inc.
    +#
    +# Licensed under the Apache License, Version 2.0 (the "License");
    +# you may not use this file except in compliance with the License.
    +# You may obtain a copy of the License at
    +#
    +#     http://www.apache.org/licenses/LICENSE-2.0
    +#
    +# Unless required by applicable law or agreed to in writing, software
    +# distributed under the License is distributed on an "AS IS" BASIS,
    +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    +# See the License for the specific language governing permissions and
    +# limitations under the License.
    +
    +- name: Set the packages to install
    +  set_fact:
    +    neutron_optional_combined_pip_packages: |-
    +      {% set packages = neutron_optional_opencontrail_pip_packages %}
    +      {{ packages }}
    +
    +- name: Install OpenContrail pip packages
    +  pip:
    +    name: "{{ neutron_optional_combined_pip_packages }}"
    +    state: "{{ neutron_pip_package_state }}"
    +    virtualenv: "{{ neutron_bin | dirname }}"
    +    virtualenv_site_packages: "no"
    +    extra_args: >-
    +      {{ neutron_developer_mode | ternary(pip_install_developer_constraints | default('--constraint /opt/developer-pip-constraints.txt'), '') }}
    +      {{ (pip_install_upper_constraints is defined) | ternary('--constraint ' + pip_install_upper_constraints | default(''),'') }}
    +      {{ pip_install_options | default('') }}
    +  register: install_packages
    +  until: install_packages|success
    +  retries: 5
    +  delay: 2
    +  tags:
    +    - opencontrail-install
    +    - opencontrail-pip-packages
    +
    +- name: Install git
    +  apt:
    +    name: git
    +    state: present
    +  delegate_to: "{{ item }}"
    +  with_items:
    +    - "{{ groups['neutron_server'] }}"
    +  tags:
    +    - opencontrail-install
    +
    +- name: Clone contrail neutron plugin
    +  git:
    +    repo: "{{ opencontrail_plugin_git_repo }}"
    +    version: "{{ opencontrail_plugin_git_install_branch }}"
    +    dest: /opt/contrail-neutron-plugin
    +    force: yes
    +  register: contrail_plugin_git_clone
    +  delegate_to: "{{ item }}"
    +  with_items:
    +    - "{{ groups['neutron_server'] }}"
    +  until: contrail_plugin_git_clone|success
    +  retries: 5
    +  delay: 2
    +  tags:
    +    - opencontrail-install
    +
    +# (jamesdenton) Will need to eventually compile and/or extract from Docker container
    +# The tasks marked (temp) should be reworked
    +
    +- name: Download Contrail python libraries (temp)
    +  vars:
    +  - dlpath: https://github.com/busterswt/contrail-openstack/raw/master
    +  get_url:
    +    url: "{{ dlpath }}/{{ item }}"
    +    dest: /opt
    +    mode: 0440
    +  with_items:
    +    - contrail-openstack-neutron-init.tar
    +  tags:
    +    - opencontrail-install
    +
    +- name: Unpack Contrail python libraries (temp)
    +  unarchive:
    +    remote_src: yes
    +    src: /opt/contrail-openstack-neutron-init.tar
    +    dest: /openstack/venvs/neutron-{{ neutron_venv_tag }}/lib/python2.7/site-packages
    +  when:
    +    inventory_hostname == groups['neutron_server'][0]
    +  tags:
    +    - opencontrail-install
    +
    +- name: Install contrail neutron plugin into venv
    +  command: "/openstack/venvs/neutron-{{ neutron_venv_tag }}/bin/python setup.py install"
    +  args:
    +    chdir: /opt/contrail-neutron-plugin
    +  delegate_to: "{{ item }}"
    +  with_items:
    +    - "{{ groups['neutron_server'] }}"
    +  tags:
    +    - opencontrail-install
    diff --git a/templates/plugins/opencontrail/ContrailPlugin.ini.j2 b/templates/plugins/opencontrail/ContrailPlugin.ini.j2
    new file mode 100644
    index 0000000..9d645b0
    --- /dev/null
    +++ b/templates/plugins/opencontrail/ContrailPlugin.ini.j2
    @@ -0,0 +1,23 @@
    +# {{ ansible_managed }}
    +
    +{% if neutron_plugin_type == 'opencontrail' %}
    +[APISERVER]
    +api_server_ip = {{ opencontrail_api_vip_address }}
    +api_server_port = {{ opencontrail_api_vip_port }}
    +multi_tenancy = True
    +contrail_extensions = ipam:neutron_plugin_contrail.plugins.opencontrail.contrail_plugin_ipam.NeutronPluginContrailIpam,policy:neutron_plugin_contrail.plugins.opencontrail.contrail_plugin_policy.NeutronPluginContrailPolicy,route-table:neutron_plugin_contrail.plugins.opencontrail.contrail_plugin_vpc.NeutronPluginContrailVpc,contrail:None,service-interface:None,vf-binding:None
    +
    +[COLLECTOR]
    +analytics_api_ip = {{ opencontrail_collector_vip_address }}
    +analytics_api_port = {{ opencontrail_collector_vip_port }}
    +
    +[keystone_authtoken]
    +auth_host = {{ internal_lb_vip_address }}
    +auth_port = {{ keystone_service_port }}
    +auth_protocol = {{ keystone_service_proto }}
    +admin_user = {{ keystone_admin_user_name }}
    +admin_password = {{ keystone_auth_admin_password }}
    +admin_tenant_name = {{ keystone_admin_tenant_name }}
    +insecure = True
    +region_name = {{ keystone_service_region }}
    +{% endif %}
    

    对现有文件的更改包括:

    root@aio1:/etc/ansible/roles/os_neutron# git diff
    diff --git a/defaults/main.yml b/defaults/main.yml
    index 162e933..7054c96 100644
    --- a/defaults/main.yml
    +++ b/defaults/main.yml
    @@ -63,6 +63,8 @@ networking_bgpvpn_git_repo: https://git.openstack.org/openstack/networking-bgpvp
     networking_bgpvpn_git_install_branch: master
     openstack_ceilometer_git_repo: https://git.openstack.org/openstack/ceilometer
     openstack_ceilometer_git_install_branch: master
    +opencontrail_plugin_git_repo: https://github.com/Juniper/contrail-neutron-plugin
    +opencontrail_plugin_git_install_branch: master
    
     # Developer mode
     neutron_developer_mode: false
    @@ -164,6 +166,7 @@ neutron_sriov_nic_agent_ini_overrides: {}
     neutron_sriov_nic_agent_init_overrides: {}
     neutron_vpn_agent_init_overrides: {}
     neutron_vpnaas_agent_ini_overrides: {}
    +neutron_opencontrail_conf_ini_overrides: {}
    
     ###
     ### Quotas
    @@ -434,3 +437,12 @@ ovs_nsh_support: False
    
     # Set higher priority to mardim PPA when ovs_nsh_support is True
     ovs_nsh_apt_pinned_packages: [{ package: "*", release: "LP-PPA-mardim-mardim-ppa"}]
    +
    +###
    +### Contrail/OpenContrail/Tungsten Fabric Configuration
    +###
    +
    +opencontrail_api_vip_address: "{{ external_lb_vip_address }}"
    +opencontrail_api_vip_port: "8082"
    +opencontrail_collector_vip_address: "{{ external_lb_vip_address }}"
    +opencontrail_collector_vip_port: "8081"
    diff --git a/templates/neutron.conf.j2 b/templates/neutron.conf.j2
    index 83d25a7..dd755ca 100644
    --- a/templates/neutron.conf.j2
    +++ b/templates/neutron.conf.j2
    @@ -42,6 +42,10 @@ core_plugin = {{ neutron_plugin_core }}
     {% if neutron_plugin_type.split('.')[0] == 'ml2' %}
     service_plugins = {{ neutron_plugin_loaded_base | join(',') }}
     {% endif %}
    +{% if neutron_plugin_type == 'opencontrail' %}
    +service_plugins = neutron_plugin_contrail.plugins.opencontrail.loadbalancer.v2.plugin.LoadBalancerPluginV2
    +api_extensions_path = /openstack/venvs/neutron-{{ neutron_venv_tag }}/lib/python2.7/site-packages/neutron_plugin_contrail/extensions:/openstack/venvs/neutron-{{ neutron_venv_tag }}/lib/python2.7/site-packages/neutron_lbaas/extensions
    +{% endif %}
    
     # MAC address generation for VIFs
     base_mac = fa:16:3e:00:00:00
    @@ -94,8 +98,9 @@ rpc_workers = {{ neutron_rpc_workers }}
    
     {% set dhcp_agents_max = num_agent if num_agent > 2 else 2 %}
     # DHCP
    -{% if neutron_plugin_type == 'ml2.dragonflow' %}
    -# In dragonflow, DHCP is fully distributed, and DHCP agents are not used
    +{% if neutron_plugin_type == ('ml2.dragonflow' or 'opencontrail') %}
    +# In dragonflow and opencontrail, DHCP is fully distributed and DHCP
    +# agents are not used
     dhcp_agent_notification = False
     {% else %}
     dhcp_agent_notification = True
    diff --git a/vars/main.yml b/vars/main.yml
    index cef4ee8..2d1c2a2 100644
    --- a/vars/main.yml
    +++ b/vars/main.yml
    @@ -121,6 +121,10 @@ neutron_plugins:
         plugin_ini: plugins/ml2/ml2_conf.ini
         driver_interface: "openvswitch"
         l3_agent_mode: "legacy"
    +  opencontrail:
    +    plugin_core: neutron_plugin_contrail.plugins.opencontrail.contrail_plugin.NeutronPluginContrailCoreV2
    +    plugin_ini: plugins/opencontrail/ContrailPlugin.ini
    +    plugin_conf_ini_overrides: "{{ neutron_opencontrail_conf_ini_overrides }}"
    
     ###
     ### ML2 Plugin Configuration
    diff --git a/vars/source_install.yml b/vars/source_install.yml
    index a246a45..24e57ea 100644
    --- a/vars/source_install.yml
    +++ b/vars/source_install.yml
    @@ -96,6 +96,13 @@ neutron_proprietary_nuage_pip_packages:
       - nuage-openstack-neutronclient
       - nuagenetlib
    
    +neutron_optional_opencontrail_pip_packages:
    +  - bitarray
    +  - bottle
    +  - geventhttpclient
    +  - psutil>=0.6.0
    +  - requests>=1.1.0
    +
     neutron_developer_constraints:
       - "git+{{ neutron_git_repo }}@{{ neutron_git_install_branch }}#egg=neutron"
       - "git+{{ neutron_fwaas_git_repo }}@{{ neutron_fwaas_git_install_branch }}#egg=neutron-fwaas"
    

    os_nova

    root@aio1:/etc/ansible/roles/os_nova# git diff
    diff --git a/defaults/main.yml b/defaults/main.yml
    index 67d92e9..bc44511 100644
    --- a/defaults/main.yml
    +++ b/defaults/main.yml
    @@ -325,6 +325,9 @@ nova_network_services:
       calico:
         use_forwarded_for: True
         metadata_proxy_enabled: False
    +  opencontrail:
    +    use_forwarded_for: True
    +    metadata_proxy_enabled: True
    
     # Nova quota
     nova_quota_cores: 20
    

    HAproxy更改

    为了提供单一的TF API和仪表板终端,我决定创建一个VIP,以平衡TF API和analytics服务之间的流量。这是否为最佳做法还有待观察,但是此处对group_vars的更改有助于创建VIP:

    root@aio1:/opt/openstack-ansible# git diff
    diff --git a/inventory/group_vars/haproxy/haproxy.yml b/inventory/group_vars/haproxy/haproxy.yml
    index b837443..dc53ef4 100644
    --- a/inventory/group_vars/haproxy/haproxy.yml
    +++ b/inventory/group_vars/haproxy/haproxy.yml
    @@ -36,6 +36,7 @@ haproxy_rabbitmq_management_whitelist_networks: "{{ haproxy_whitelist_networks }
     haproxy_repo_git_whitelist_networks: "{{ haproxy_whitelist_networks }}"
     haproxy_repo_cache_whitelist_networks: "{{ haproxy_whitelist_networks }}"
     haproxy_opendaylight_whitelist_networks: "{{ haproxy_whitelist_networks }}"
    +haproxy_opencontrail_whitelist_networks: "{{ haproxy_whitelist_networks }}"
    
     haproxy_default_services:
       - service:
    @@ -365,3 +366,23 @@ haproxy_default_services:
           haproxy_backend_httpcheck_options:
             - expect status 405
           haproxy_service_enabled: "{{ (groups['ceph-rgw'] is defined and groups['ceph-rgw'] | length > 0) or (ceph_rgws | length > 0) }}"
    +  - service:
    +      haproxy_service_name: opencontrail-api
    +      haproxy_backend_nodes: "{{ groups['opencontrail-api_hosts'] | default([]) }}"
    +      haproxy_bind: "{{ [opencontrail_api_vip_address] }}"
    +      haproxy_port: 8082
    +      haproxy_balance_type: tcp
    +      haproxy_timeout_client: 5000s
    +      haproxy_timeout_server: 5000s
    +      haproxy_whitelist_networks: "{{ haproxy_opencontrail_whitelist_networks }}"
    +      haproxy_service_enabled: "{{ neutron_plugin_type == 'opencontrail' }}"
    +  - service:
    +      haproxy_service_name: opencontrail-collector
    +      haproxy_backend_nodes: "{{ groups['opencontrail-analytics_hosts'] | default([]) }}"
    +      haproxy_bind: "{{ [opencontrail_collector_vip_address] }}"
    +      haproxy_port: 8081
    +      haproxy_balance_type: tcp
    +      haproxy_timeout_client: 5000s
    +      haproxy_timeout_server: 5000s
    +      haproxy_whitelist_networks: "{{ haproxy_opencontrail_whitelist_networks }}"
    +      haproxy_service_enabled: "{{ neutron_plugin_type == 'opencontrail' }}"
    

    我在All-in-One节点上使用此方法遇到的一些问题,包括HAproxy将VIP绑定到端口8081的能力。后来在过程中发现,Tungsten Fabric playbooks在0.0.0.0:8081上创建了一个侦听器,从而阻止了VIP绑定在同一端口上。这里的替代方法是注释掉该服务,或在部署HAproxy后将其禁用。对于HAproxy在不同节点上的多节点安装,可以保留它。最终,这两种服务的负载平衡可能无法实现,但我暂时将其保留。

    环境更改

    默认的Neutron env.d框架将部署原本没必要部署的Neutron代理容器。通过覆盖默认值,我们可以删除代理容器并定义一些新的组件:

    cat <> /etc/openstack_deploy/env.d/neutron.yml
    component_skel:
      neutron_server:
        belongs_to:
          - neutron_all
      opencontrail_vrouter:
        belongs_to:
        - neutron_all
      opencontrail_api:
        belongs_to:
        - neutron_all
      opencontrail_analytics:
        belongs_to:
        - neutron_all
    
    container_skel:
      neutron_agents_container:
        contains: {}
      opencontail-vrouter_container:
        belongs_to:
          - compute_containers
        contains:
          - opencontrail_vrouter
        properties:
          is_metal: true
      opencontrail-api_container:
        belongs_to:
          - opencontrail-api_containers
        contains:
          - opencontrail_api
        properties:
          is_metal: true
      opencontrail-analytics_container:
        belongs_to:
          - opencontrail-analytics_containers
        contains:
          - opencontrail_analytics
        properties:
          is_metal: true
      neutron_server_container:
          belongs_to:
            - network_containers
          contains:
            - neutron_server
    
    physical_skel:
      opencontrail-api_containers:
        belongs_to:
          - all_containers
      opencontrail-api_hosts:
        belongs_to:
          - hosts
      opencontrail-analytics_containers:
        belongs_to:
          - all_containers
      opencontrail-analytics_hosts:
        belongs_to:
          - hosts
    EOF
    

    在这一点上,我不确定所有这些更改都是必要的,以及方法是否正确,但目前暂时这样。
    在 openstack_user_config.yml文件中,我定义了两个新的组,以便能够在主机之间拆分API和Analytics服务。由于这是一个AlO节点,因此IP是相同的:

    opencontrail-api_hosts:
      aio1:
        ip: 172.29.236.100
    
    opencontrail-analytics_hosts:
      aio1:
        ip: 172.29.236.100
    

    Overrides

    os_neutron角色的更改导致添加了新的默认值,并且还需要一些overrides。在user_variables.yml中添加了以下内容:

    neutron_plugin_type: opencontrail
    neutron_driver_quota: neutron_plugin_contrail.plugins.opencontrail.quota.driver.QuotaDriver
    opencontrail_plugin_git_install_branch: R5.0
    

    角色中指定的一些默认值包括:

    opencontrail_api_vip_address: "{{ external_lb_vip_address }}"
    opencontrail_api_vip_port: "8082"
    opencontrail_collector_vip_address: "{{ external_lb_vip_address }}"
    opencontrail_collector_vip_port: "8081"
    opencontrail_plugin_git_repo: https://github.com/Juniper/contrail-neutron-plugin
    opencontrail_plugin_git_install_branch: master
    neutron_opencontrail_conf_ini_overrides: {}
    

    最终要求仍尚未确定。

    运行OpenStack-Ansible playbooks

    至此,已经完成了对OpenStack-Ansible playbooks的大部分更改,并且可以部署OpenStack了:

    # cd /opt/openstack-ansible/playbooks
    # openstack-ansible setup-hosts.yml
    # openstack-ansible setup-infrastructure.yml
    # openstack-ansible setup-openstack.yml
    

    虽然Neutron会被下线,但在使用网络之前,还需要进行一些其它更改并部署Tungsten Fabric。可以使用Horizon,但可能无法完全发挥它的功能。
    接下来,我们必须要执行一些操作,包括克隆Juniper ansible playbook repo,设置overrides,以及运行这些playbooks,以安装Tungsten Fabric,并为与TF相关的服务overlay一些Docker容器。

    部署Tungsten Fabric

    到这里,我们已经部署了OpenStack,并且知道已实现的网络后端,但是该后端还不存在。Juniper提供的playbooks可以使用Docker容器部署Tungsten Fabric。这些相同的playbooks还可以部署基于Kolla的OpenStack版本,这是另一种OpenStack部署策略。在这里,我们实际上只需要与Tungsten Fabric相关的代码。
    为了实现一些更改,我克隆了repo,并进行了部署在基于OpenStack-Ansible的云上所需的必要更改。
    克隆repo:

    # git clone http://github.com/busterswt/contrail-ansible-deployer /opt/openstack-ansible/playbooks/contrail-ansible-deployer
    # cd /opt/openstack-ansible/playbooks/contrail-ansible-deployer
    # git checkout osa
    

    Overrides

    Juniper playbooks依赖于它们自己的清单和overrides,因此执行以下几步可能会感觉有些多余。我还没有重新编写playbooks以利用OpenStack-Ansible的可用清单。
    可以在/etc/openstack_deploy/user_opencontrail_vars.yml的新文件中定义这些overrides:

    config_file: /etc/openstack_deploy/user_opencontrail_vars.yml
    opencontrail_api_vip_address: ""
    opencontrail_collector_vip_address: ""
    provider_config:
      bms:
        ssh_pwd:
        ssh_user: root
        ssh_public_key: /root/.ssh/id_rsa.pub
        ssh_private_key: /root/.ssh/id_rsa
        ntpserver: 129.6.15.28
    instances:
      aio1:
        provider: bms
        ip: 172.29.236.100
        roles:
          config_database:
          config:
          control:
          analytics_database:
          analytics:
          webui:
          vrouter:
    	    VROUTER_GATEWAY: 10.50.0.1
    		PHYSICAL_INTERFACE: ens160
    global_configuration:
      CONTAINER_REGISTRY: opencontrailnightly
    #  CONTAINER_REGISTRY: hub.juniper.net/contrail
    #  CONTAINER_REGISTRY_USERNAME: 
    #  CONTAINER_REGISTRY_PASSWORD: 
    contrail_configuration:
      CLOUD_ORCHESTRATOR: openstack
      CONTRAIL_VERSION: latest
    #  CONTRAIL_VERSION: 5.0.0-0.40
    # UPGRADE_KERNEL: true
      UPGRADE_KERNEL: false
      KEYSTONE_AUTH_HOST: ""
      KEYSTONE_AUTH_PUBLIC_PORT: 5000
      KEYSTONE_AUTH_PROTO: http
      KEYSTONE_AUTH_URL_VERSION:
      KEYSTONE_AUTH_ADMIN_USER: admin
      KEYSTONE_AUTH_ADMIN_PASSWORD: ""
      KEYSTONE_AUTH_URL_VERSION: /v3
      CONTROLLER_NODES: 172.29.236.100
      CONTROL_NODES: 172.29.236.100
      ANALYTICSDB_NODES: 172.29.236.100
      WEBUI_NODES: 172.29.236.100
      ANALYTICS_NODES: 172.29.236.100
      CONFIGDB_NODES: 172.29.236.100
      CONFIG_NODES: 172.29.236.100
    

    请特别注意以下键/值对:

    VROUTER_GATEWAY: 10.50.0.1
    PHYSICAL_INTERFACE: ens160
    
    CONTAINER_REGISTRY: opencontrailnightly
    #  CONTAINER_REGISTRY: hub.juniper.net/contrail
    #  CONTAINER_REGISTRY_USERNAME: 
    #  CONTAINER_REGISTRY_PASSWORD: 
    CONTRAIL_VERSION: latest
    #  CONTRAIL_VERSION: 5.0.0-0.40
    
    CONTROLLER_NODES: 172.29.236.100
    CONTROL_NODES: 172.29.236.100
    ANALYTICSDB_NODES: 172.29.236.100
    WEBUI_NODES: 172.29.236.100
    ANALYTICS_NODES: 172.29.236.100
    CONFIGDB_NODES: 172.29.236.100
    CONFIG_NODES: 172.29.236.100
    

    第一部分定义了vRouter所需的vhost0 接口将重新利用或者利用的“physical”接口。这是一个具有单个NIC的AIO节点,这意味着机会很不错,playbooks会自动确定要使用的接口。我们继续前进,同时在此处进行了标注。该主机的IP地址是10.50.0.221,网关是10.50.0.1。对于多宿主主机,有可能将接口及其自己的网关地址专用于vRouter。我尚未进行多NIC部署,但期待能实现。

    第二部分定义了将从中下载容器的Docker注册表。这里opencontailnightly 注册表包含每隔一到两天生成的容器,这些容器不一定在任何给定的日期都是有效的。如果你通过Juniper访问GA注册表,还可以定义该注册表并提供访问凭据。在nightly注册表中唯一可用的版本是latest版本,而Juniper注册表可能具有已标记的发行版。记得要使用适当的注册表。

    第三部分定义了各个节点类型的地址,用于在整个playbooks中用于将值插入模板。这些是当前成功完成playbooks所必需的。请注意,我在这里使用了“CONTAINER_NET”地址,是希望Tungsten Fabric和OpenStack可以在LXC使用的现有容器网络上进行通信。在此AIO节点中,172.29.236.100是br-mgmt上配置的IP,并且恰好也是内部VIP地址。部署在Docker容器中的服务会将端口绑定到其各自服务的IP上。在多节点的安装中,此更改的方式是否可行仍为待定。

    OSA repo更改

    OpenStack-Ansible包括用于软件包管理repo服务器,该服务器目前禁止安装部署Tungsten Fabric所需的某些版本的软件包。在这里,我通过修改/root/.pip/pip.conf禁用了内部repo服务器:

    [global]
    disable-pip-version-check = true
    timeout = 120
    #index-url = http://172.29.236.100:8181/simple
    #trusted-host =
    #        172.29.236.100
    
    [install]
    upgrade = true
    upgrade-strategy = only-if-needed
    pre = true
    

    运行playbooks

    到这里,我们应该有了一个很好的起点,可以开始运行TF playbooks了。
    首先,节点必须运行引导程序:

    # cd /opt/openstack-ansible/playbooks/
    # openstack-ansible -e orchestrator=openstack contrail-ansible-deployer/playbooks/configure_instances.yml
    

    UPGRADE_KERNEL 为true时,主机可能会重新启动。如果发生这种情况,请再次重新运行该playbook。如果TF节点不是需要部署的主机,则playbooks具有一个计时器,等待主机返回。
    接下来,运行install_contrail.yml playbook以部署TF:

    # cd /opt/openstack-ansible/playbooks/
    # openstack-ansible -e orchestrator=openstack contrail-ansible-deployer/playbooks/install_contrail.yml
    

    安装其它组件

    由于Juniper playbooks期望基于Kolla的部署,因此某些组件未安装在基于OpenStack-Ansible的云之上。我自己从Docker容器中提取了一些实用程序、模块,以及更多的内容,并编写了一个playbook来实现这些内容:

    # cd /opt/openstack-ansible/playbooks/
    # openstack-ansible contrail-ansible-deployer/playbooks/install_contrailtools.yml
    

    重启

    一旦TF完成部署,vRouter内核模块已编译并插入后,可能有必要重启主机以清除vRouter代理容器内遇到的一些问题,包括:

    contrail-vrouter-agent: controller/src/vnsw/agent/vrouter/ksync/ksync_memory.cc:107: void KSyncMemory::Mmap(bool): Assertion `0' failed.
    /entrypoint.sh: line 304: 28142 Aborted                 (core dumped) $@
    

    我发现在安装后重新启动主机,足以解决这些问题。

    部署后的工作

    现在我们已经安装了Tungsten Fabric,可以使用 contrail-status命令检查服务的状态:

    root@aio1:/opt/openstack-ansible/playbooks# contrail-status
    Pod        Service         Original Name                          State    Status
    analytics  api             contrail-analytics-api                 running  Up 21 minutes
    analytics  collector       contrail-analytics-collector           running  Up 21 minutes
    analytics  nodemgr         contrail-nodemgr                       running  Up 21 minutes
    analytics  query-engine    contrail-analytics-query-engine        running  Up 21 minutes
    config     api             contrail-controller-config-api         running  Up 24 minutes
    config     cassandra       contrail-external-cassandra            running  Up 27 minutes
    config     device-manager  contrail-controller-config-devicemgr   running  Up 24 minutes
    config     nodemgr         contrail-nodemgr                       running  Up 24 minutes
    config     rabbitmq        contrail-external-rabbitmq             running  Up 27 minutes
    config     schema          contrail-controller-config-schema      running  Up 24 minutes
    config     svc-monitor     contrail-controller-config-svcmonitor  running  Up 24 minutes
    config     zookeeper       contrail-external-zookeeper            running  Up 27 minutes
    control    control         contrail-controller-control-control    running  Up 23 minutes
    control    dns             contrail-controller-control-dns        running  Up 23 minutes
    control    named           contrail-controller-control-named      running  Up 23 minutes
    control    nodemgr         contrail-nodemgr                       running  Up 23 minutes
    database   cassandra       contrail-external-cassandra            running  Up 22 minutes
    database   nodemgr         contrail-nodemgr                       running  Up 22 minutes
    database   zookeeper       contrail-external-zookeeper            running  Up 22 minutes
    vrouter    agent           contrail-vrouter-agent                 running  Up 19 minutes
    vrouter    nodemgr         contrail-nodemgr                       running  Up 19 minutes
    webui      job             contrail-controller-webui-job          running  Up 23 minutes
    webui      web             contrail-controller-webui-web          running  Up 23 minutes
    
    vrouter kernel module is PRESENT
    == Contrail control ==
    control: active
    nodemgr: active
    named: active
    dns: active
    
    == Contrail database ==
    nodemgr: active
    zookeeper: active
    cassandra: active
    
    == Contrail analytics ==
    nodemgr: active
    api: initializing (UvePartitions:UVE-Aggregation[Partitions:0] connection down)
    collector: initializing (KafkaPub:172.29.236.100:9092 connection down)
    query-engine: active
    
    == Contrail webui ==
    web: active
    job: active
    
    == Contrail vrouter ==
    nodemgr: active
    agent: active
    
    == Contrail config ==
    api: active
    zookeeper: active
    svc-monitor: active
    nodemgr: active
    device-manager: active
    cassandra: active
    rabbitmq: active
    schema: active
    

    如果看到nodemgr 服务处于Initializing 状态(NTP状态未同步),请尝试重启主机上的NTP。你可能需要定义多个NTP服务器。我尚未解决Analytics的问题,希望能尽快解决。

    一些问题

    我想说的是,到这里一切都应该可以正常进行了,但事实并非如此!使用OpenStackNeutron客户端可能无法正常工作,因为neutron-server服务可能因以下错误而运行失败:

    Unrecoverable error: please check log for details.: ExtensionsNotFound: Extensions not found: ['route-table']. Related to WARNING neutron.api.extensions [-] Extension file vpcroutetable.py wasn't loaded due to cannot import name attributes.
    

    在Rocky中,一个TF插件所依赖而被OpenStack弃用的模块被删除了。不要担心,它只是移动到了另一个模块/文件中。
    在 neutron_server容器中,我修改了/openstack/venvs/neutron-18.0.0.0b3/lib/python2.7/site-packages/neutron_plugin_contrail/extensions/vpcroutetable.py文件,如下所示:

    -from neutron.api.v2 import attributes as attr
    +from neutron_lib.api import attributes as attr
    

    需要重新启动neutron-server服务:

    # systemctl restart neutron-server
    

    如何在OpenStack-Ansible上集成Tungsten Fabric(下)


Log in to reply