Test matrix cloud-regionsrv-client

Test one SLE 15, SLE 16 and one SLE 12 based instance

The first step is to disable the license-watcher. If not disabled it will
interfer with the test.

- systemctl disable guestregister-lic-watcher.timer
- systemctl stop guestregister-lic-watcher.timer

on-demand
After installing the test package with "zypper in"
- run /usr/sbin/updatesmtcache
  + $? is 0
- run /usr/sbin/createregioninfo
  + $? is 0

- Initial boot
  + registration success
    - check log
  + zypper lr has repos
  + SUSEConnect -s shows the registration status
  + SUSEConnect -l shows a list of modules/extensions available or installed
  + instance-flavor-check return PAYG/BYOS accordingly
  On SLE 15 and later
  + grep REGISTRY_AUTH_FILE /etc/profile.local has value
  + grep DOCKER_CONFIG /etc/profile.local has value
  + grep susecloud /etc/docker/daemon.json has value
  + grep susecloud /etc/containers/registries.conf has value
- registercloudguest --clean
  + no error no message
  + zypper lr has no repos
    ~ On google there is a system repo pointing to Google, that's
      expected and OK
  + ls /etc/zypp/credentials.d returns empty
  + ls /etc/zypp/services.d/ returns empty
  + ls /var/cache/cloudregister/* returns empty or dir does not exist
  + grep susecloud /etc/hosts returns empty
  + instance-flavor-check return PAYG/BYOS accordingly
  On SLE 15 and later
  + grep REGISTRY_AUTH_FILE /etc/profile.local returns empty
  + grep DOCKER_CONFIG /etc/profile.local returns empty
  + cat /etc/docker/daemon.json
    - does not exists or has no reference to susecloud
  + cat /etc/containers/registries.conf
    - does not exists or has no reference to susecloud
- registercloudguest
  + no error
  + success message on stdout
  + $? is 0
  + zypper lr has repos
  On SLE 15 and later
  + cat /etc/docker/daemon.json
    - Should have registry for the update infrastructure and registry.suse.com
  + cat /etc/containers/registries.conf
    - Should have registry for the update infrastructure and registry.suse.com
  + grep REGISTRY_AUTH_FILE /etc/profile.local
  + grep DOCKER_CONFIG /etc/profile.local
  + source /etc/profile.local
  + systemctl start docker.service
  + systemctl status docker.service
    - Service should be running
  + docker pull bci/bci-base
    - Should pull the container from registry.suse.com
  + systemctl stop docker.service
  + zypper in podman
  + podman pull bci/bci-base
  # The next test will only work if the pulled BCI container version matches
  # the version of the underlying OS, for example a 15 SP6 container on
  # a 15 SP6 host.
  # If it fails ignore if you are certain it is not broken because of changes
  # in this code
  + zypper in libcontainers-sles-mounts
  + systemctl start containerbuild-regionsrv.service
  + systemctl status containerbuild-regionsrv.service
  + podman run --network host -it bci/bci-base
  + zypper lr should have SLES repos; exit the container
  + exit the container
- registercloudguest --clean
  + zypper lr has no repos
  On SLE 15 and later
  + cat /etc/docker/daemon.json
    - File does not exist or
    - No reference to registry on update infrastructure and registry.suse.com
  + cat /etc/containers/registries.conf
    - File does not exist or
    - No reference to registry on update infrastructure and registry.suse.com
  + grep REGISTRY_AUTH_FILE /etc/profile.local
    - Turns up empty
  + grep DOCKER_CONFIG /etc/profile.local
    - Turns up empty
- registercloudguest
- registercloudguest --force-new
  + no error
  + success message on stdout
  + $? is 0
  + zypper lr has repos
  On SLE 15 and later
  + cat /etc/docker/daemon.json
    - Should have registry for the update infrastructure and registry.suse.com
  + cat /etc/containers/registries.conf
    - Should have registry for the update infrastructure and registry.suse.com
  + grep REGISTRY_AUTH_FILE /etc/profile.local
  + grep DOCKER_CONFIG /etc/profile.local
- rm /etc/pki/trust/anchors/registration_server_*.pem
  + update-ca-certificates
  + registercloudguest --force-new
  + success message on stdout
  + $? is 0
  + zypper lr has repos
- Update server failover
  + ip=`grep -m 1 susecloud.net /etc/hosts | cut -f1`
  + iptables -A OUTPUT -d $ip -j DROP
  + zypper ref
  + grep -i equivalent /var/log/cloudregister
  + grep susecloud.net /etc/hosts
  + IP addresses in the last two outputs should match

byos - test on SLES4SAP instance
After installing the test package with "zypper in"
- run /usr/sbin/updatesmtcache
  + $? is 0
- run /usr/sbin/createregioninfo
  + $? is 0

- Initial boot
  + no registration
  + on Azure regionsrv-enabler-azure.timer is enabled
  + guestregister.service is not enabled and not running
  + /var/cache/cloudregister/ only has framework_info file or is empty
  + zypper lr has no repos
- SUSEConnect -d
  + produce error message
- registercloudguest --clean
  + no error no message
  + zypper lr has no repos
- SUSEConnect -r XXX
  + registration successful
- SUSEConnect -p sle-module-public-cloud/$VERSION/x86_64
  + registration successful
- SUSEConnect -d -p sle-module-public-cloud/$VERSION/x86_64
  + module deletion successful
  + Requires SUSEConnect > 0.3.32
- registercloudguest --clean
  + $? is 0
  + no error no message
  + zypper lr has no repos
  + nothing in /etc/zypp/credentials.d
  + nothing in /etc/zypp/services.d/
- registercloudguest -r XXX
  + no error
  + sucees message on stdout
  + $? is 0
  + repos include HA
- registercloudguest --force-new -r XXX
  + no error
  + sucees message on stdout
  + $? is 0
  + repos include HA
- SUSEConnect -p sle-module-public-cloud/$VERSION/x86_64
  + registration successful
  + Cloud based RMT server is the target
- registercloudguest --clean

Migration testing and LTSS testing
SLE 15
- Start a PAYG instance 2 SPs back from the current one
- Check the instance is properly registered
- zypper up
  + Expected to succeed
  With the new cloud-regionsrv-client code in place
- registercloudguest --clean
- registercloudguest -r LTSS_REG_CODE
  + Expected to fail because the underlying system base product is not
    registered
- registercloudguest
  + no error
  + sucees message on stdout
  + $? is 0
- registercloudguest -r LTSS_REG_CODE
  + zypper lr | grep LTSS has LTSS repositories
  + zypper ref
  + zypper up
    + Expected to have new updates due to the addition of the LTSS repos
  + source /etc/profile.local
  + systemctl start docker.service
  + systemctl status docker.service
    - Service should be running
  + zypper in podman
  + podman search --list-tags registry-$CSP.susecloud.net/suse/ltss/sle$VERSION.SPNUMBER/sle15
  + podman pull suse/ltss/sle$VERSION.SPNUMBER/sle15:TAGNUMBER
    - Should pull the container from registry-$FRAMEWORK.susecloud.net
  + docker pull suse/ltss/sle$VERSION/bci-base
  + systemctl stop docker.service
- zypper migration
  + Should fail because LTSS repos are active, and possibly other modules
  + Use the command presented to remove the product/modules
- zypper migration
  + Migrate to the highest available SP
  + Expected to succeed
- Start a BYOS instance 2 SPs back from the current one
  With the new cloud-regionsrv-client code in place
- registercloudguest -r XXX
  + no error
  + sucees message on stdout
- Add the Public Cloud Module if it is not setup
- zypper up
  + Expected to succeed
- registercloudguest -r LTSS_REG_CODE
  + zypper lr | grep LTSS has LTSS repositories
  + zypper ref
  + zypper up
    + Expected to have new updates due to the addition of the LTSS repos
  + source /etc/profile.local
  + systemctl start docker.service
  + systemctl status docker.service
    - Service should be running
  + zypper in podman
  + podman search --list-tags registry-ec2.susecloud.net/suse/ltss/sle$VERSION.SPNUMBER/sle15
  + podman pull suse/ltss/sle$VERSION.SPNUMBER/sle15:TAGNUMBER
    - Should pull the container from registry-$FRAMEWORK.susecloud.net
  + docker pull suse/ltss/sle$VERSION.SPNUMBER/bci-base:TAGNUMBER
    - Should pull the container from registry-$FRAMEWORK.susecloud.net
  + systemctl stop docker.service
- zypper migration
  + Should fail because LTSS repos are active, and possibly other modules
  + Use the command presented to remove the product/modules
- zypper migration
  + Migrate to the highest available SP
  + Expected to succeed
- zypper migration
  + Should fail because LTSS repos are active, and possibly other modules
  + Use the command presented to remove the product/modules
- zypper migration
  + Migrate to the highest available SP
  + Expected to succeed

Extended testing
- Clean out /var/log/cloudregister
- Create an image from a registered instance (PAYG)
- Start an instance from the created image in a different region
  + /var/log/cloudregister contains
    "INFO:Region change detected, registering to new servers"
  + instance has repos
  + cat /etc/hosts server point to an update server in the region
  + zypper up has no errors
