0707010015BC0B000081A40000000000000000000000015BE091E90000003E000000FD0000000200000000000000000000003E00000000python-cinderlm-0.0.2+git.1541444073.4d3347c/.copyrightignorerequirements.txt MANIFEST.in .gitignore tests/rings/samples/* 0707010015BC0E000081A40000000000000000000000015BE091E90000007C000000FD0000000200000000000000000000003800000000python-cinderlm-0.0.2+git.1541444073.4d3347c/.gitreview[gerrit] host=gerrit.suse.provo.cloud port=29418 project=ardana/cinderlm.git defaultremote=ardana defaultbranch=stable/pike 0707010015BC09000081A40000000000000000000000015BE091E90000279F000000FD0000000200000000000000000000003500000000python-cinderlm-0.0.2+git.1541444073.4d3347c/LICENSE Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. 0707010015BC53000081A40000000000000000000000015BE091E90000006E000000FD0000000200000000000000000000003900000000python-cinderlm-0.0.2+git.1541444073.4d3347c/MANIFEST.inexclude .gitignore exclude .gitreview include requirements.txt include test-requirements.txt include LICENSE 0707010015BC54000081A40000000000000000000000015BE091E9000002A9000000FD0000000200000000000000000000003700000000python-cinderlm-0.0.2+git.1541444073.4d3347c/README.md (c) Copyright 2015 Hewlett Packard Enterprise Development LP (c) Copyright 2017 SUSE LLC Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. # cinderlm - Cinder Lifecycle Managment Cinder diagnostic scripts. 0707010015BBFC000041ED0000000000000000000000035BE091E900000000000000FD0000000200000000000000000000003600000000python-cinderlm-0.0.2+git.1541444073.4d3347c/cinderlm0707010015BC06000081A40000000000000000000000015BE091E900000000000000FD0000000200000000000000000000004200000000python-cinderlm-0.0.2+git.1541444073.4d3347c/cinderlm/__init__.py0707010015BC08000081A40000000000000000000000015BE091E900001E21000000FD0000000200000000000000000000004F00000000python-cinderlm-0.0.2+git.1541444073.4d3347c/cinderlm/cinder_capacity_check.py#!/usr/bin/env python # # (c) Copyright 2016 Hewlett Packard Enterprise Development LP # (c) Copyright 2017 SUSE LLC # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. # from __future__ import print_function from cinderclient.client import Client as CinderClient import ConfigParser import socket import sys import time import traceback cinderlm_conf_file = "/etc/cinderlm/cinderlm.conf" # This name is known by monasca - do NOT change MODULE_SERVICE_NAME = 'block-storage' capacity_metrics = {'cinderlm.cinder.backend.total.size': 'Total Capacity Metric', 'cinderlm.cinder.backend.total.avail': 'Total Available Capacity Metric', 'cinderlm.cinder.backend.physical.list': 'Cinder physical backend list'} def metric(name, value, dimensions, timestamp, msg=None): """Construct the metric dictionary To list these metrics (for say the last two hours): monasca measurement-list cinderlm.cinder.backend.total.size -120 \ --dimensions hostname=,backendname=, \ name= monasca measurement-list cinderlm.cinder.backend.total.avail -120 \ --dimensions hostname=,backendname=, \ name= monasca measurement-list cinderlm.cinder.backend.physical.list -120 \ --dimensions hostname=,backends=physical """ metric = { 'metric': name, 'value': value, 'dimensions': dimensions, 'timestamp': timestamp, } if msg is None: msg = capacity_metrics.get(name, 'Unknown Metric') metric['value_meta'] = {'msg': msg} return metric def get_cinder_client(): cp = ConfigParser.RawConfigParser() cp.read(cinderlm_conf_file) cinderlm_username = cp.get('DEFAULT', 'cinderlm_user') cinderlm_password = cp.get('DEFAULT', 'cinderlm_password') cinderlm_project_name = cp.get('DEFAULT', 'cinderlm_project_name') cinderlm_ca_cert_file = cp.get('DEFAULT', 'cinderlm_ca_cert_file') keystone_auth_url = cp.get('DEFAULT', 'cinderlm_auth_url') cinder_client_version = 2 return CinderClient(cinder_client_version, username=cinderlm_username, api_key=cinderlm_password, project_id=cinderlm_project_name, auth_url=keystone_auth_url, endpoint_type='internalURL', cacert=cinderlm_ca_cert_file) def _get_capacity(): cinder_client = get_cinder_client() return cinder_client.pools.list(detailed=True) def get_capacity(): results = [] backend_capacity_info = [] physical_backend_list = [] cp = ConfigParser.RawConfigParser() cp.read(cinderlm_conf_file) run_capacity_check = cp.get('DEFAULT', 'cinderlm_capacity_check') if run_capacity_check == "True": try: backend_capacity_info = _get_capacity() except Exception: t, v, tb = sys.exc_info() backtrace = ' '.join(traceback.format_exception(t, v, tb)) # Because the length of the value_meta string is limited only take # the last 1900 characters, hard limit is 2048. backtrace = backtrace.replace('\n', ' ')[-1900:] # We were expecting size and avail metrics size_data = dict(metric='cinderlm.cinder.backend.total.size', value=-1, dimensions={'service': MODULE_SERVICE_NAME, 'hostname': socket.gethostname(), 'component': 'cinder-capacity', 'name': 'undetermined', 'backendname': 'undetermined'}, value_meta={'get_capacity': backtrace}) avail_data = dict(metric='cinderlm.cinder.backend.total.avail', value=-1, dimensions={'service': MODULE_SERVICE_NAME, 'hostname': socket.gethostname(), 'component': 'cinder-capacity', 'name': 'undetermined', 'backendname': 'undetermined'}, value_meta={'get_capacity': backtrace}) physical_backend_data = dict( metric='cinderlm.cinder.backend.physical.list', value=-1, dimensions={'service': MODULE_SERVICE_NAME, 'hostname': socket.gethostname(), 'component': 'cinder-backends', 'backends': 'undetermined'}, value_meta={'get_backends': backtrace}) results.extend([size_data, avail_data, physical_backend_data]) for backend in backend_capacity_info: try: backend.total_capacity_gb = float(backend.total_capacity_gb) except ValueError: backend.total_capacity_gb = float("-1") total_size = metric('cinderlm.cinder.backend.total.size', backend.total_capacity_gb, {'service': MODULE_SERVICE_NAME, 'hostname': socket.gethostname(), 'component': 'cinder-capacity', 'name': backend.name, 'backendname': backend.volume_backend_name}, time.time()) results.append(total_size) try: backend.free_capacity_gb = float(backend.free_capacity_gb) except ValueError: backend.free_capacity_gb = float("-1") total_avail = metric('cinderlm.cinder.backend.total.avail', backend.free_capacity_gb, {'service': MODULE_SERVICE_NAME, 'hostname': socket.gethostname(), 'component': 'cinder-capacity', 'name': backend.name, 'backendname': backend.volume_backend_name}, time.time()) results.append(total_avail) # Generate the list of backends physical_backend_list.append(backend.name) physical_backend_string = ",".join(physical_backend_list) physical_backends = metric('cinderlm.cinder.backend.physical.list', len(physical_backend_list), {'service': MODULE_SERVICE_NAME, 'hostname': socket.gethostname(), 'component': 'cinder-backends', 'backends': 'physical'}, time.time(), physical_backend_string) results.append(physical_backends) return results 0707010015BBFD000081A40000000000000000000000015BE091E900004330000000FD0000000200000000000000000000004600000000python-cinderlm-0.0.2+git.1541444073.4d3347c/cinderlm/cinder_check.py#!/usr/bin/env python # # (c) Copyright 2015-2106 Hewlett Packard Enterprise Development LP # (c) Copyright 2017 SUSE LLC # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. # from __future__ import print_function import argparse import cinderclient from cinderclient.client import Client as CinderClient from datetime import datetime from novaclient.client import Client as NovaClient import os import sys import time argparser = argparse.ArgumentParser(usage="Cinder Check Utility") def create_arguments(parser): """Sets up the CLI and config-file options""" client_args = parser.add_argument_group('client test arguments') client_args.add_argument('-t', '--tenantname', default='admin', help='Tenant Name') client_args.add_argument('--interface', default='publicURL', help='public/internal/admin URL') client_args.add_argument('-u', '--username', default='admin', help='Username') client_args.add_argument('-p', '--password', default='admin', help='Password') client_args.add_argument('--cacert', default='/etc/ssl/certs/ca-certificates.crt', help='CA cert file for TLS/SSL') client_args.add_argument( '--auth_url', default='http://standard-ccp-vip-KEY-API-mgmt:35357/v2.0', help='auth_url') client_args.add_argument('-V', '--api-version', dest="api_version", default="1", type=str, help="Run check using version specified") client_args.add_argument('-N', '--nova-api-version', dest="nova_api_version", default="2", type=str, help="Run check using version specified") client_args.add_argument('-v', '--verbose', dest="verbose", default=False, action="store_true", help="Run check in verbose mode") client_args.add_argument('-a', '--check-api', dest="check_api", default=False, action="store_true", help="Run Cinder API monitoring check") client_args.add_argument('-f', '--full', dest="full", default=False, action="store_true", help="Run a more detailed check") client_args.add_argument('-i', '--image', default=None, help="Specify the image to boot an instance.") client_args.add_argument('-l', '--flavor', default=None, help="Specify the flavor to boot an instance.") class CinderCheckClient(object): def __init__(self, options): self.options = options self.creds = {'username': options.username, 'tenant_name': options.tenantname, 'password': options.password, 'auth_url': options.auth_url, 'interface': options.interface, 'cacert': options.cacert} def get_nova_client(self): return NovaClient(self.options.nova_api_version, username=self.creds['username'], api_key=self.creds['password'], project_id=self.creds['tenant_name'], auth_url=self.creds['auth_url'], endpoint_type=self.creds['interface'], cacert=self.creds['cacert']) def get_api_client(self): return CinderClient(self.options.api_version, username=self.creds['username'], api_key=self.creds['password'], project_id=self.creds['tenant_name'], auth_url=self.creds['auth_url'], endpoint_type=self.creds['interface'], cacert=self.creds['cacert']) def print(self, msg): print("%s: %s" % (datetime.utcnow().isoformat(), msg)) def run_tests(self): """Main part of program. Runs tests specified on command line.""" if self.options.check_api: self.api_tests() def api_tests(self): """Run Cinder API tests""" self.print("Cinder API tests") self.client = self.get_api_client() self.novaclient = self.get_nova_client() if self.options.api_version == '1': self.api_tests_v1() elif self.options.api_version == '2': self.api_tests_v2() def api_tests_v1(self): self.print("Test: API V1") self.api_tests_common('1') def api_tests_v2(self): self.print("Test: API V2") self.api_tests_common('2') def _name_for_vers(self, vol, vers): return (vol.display_name if vers == '1' else vol.name) def _wait_for_instance_status(self, instance_id, status_list, timeout=60): """Loop and wait for instance to reach a specified state.""" loop_counter = int(timeout) / 2 while loop_counter > 0: loop_counter -= 1 try: instance = self.novaclient.servers.get(instance_id) if instance.status not in status_list: time.sleep(2) else: break except Exception as e: raise Exception("api:INSTANCEGET #1 Failed : %s" % (e)) else: raise Exception("api:INSTANCEGET #1 timed out") self.print("Instance: %s went to state %s." % (instance_id, instance.status)) return instance.status def _wait_for_status(self, vol_id, status_list, timeout=60): """Loop and wait for volume to reach a specified state.""" loop_counter = int(timeout) / 2 while loop_counter > 0: loop_counter -= 1 try: vol = self.client.volumes.get(vol_id) if vol.status not in status_list: time.sleep(2) else: break except Exception as e: raise Exception("api:VOLGET #1 Failed : %s" % (e)) else: raise Exception("api:VOLGET #1 timed out") self.print("Volume: %s went to state %s." % (vol_id, vol.status)) return vol.status def _wait_for_backup_status(self, bck_id, status_list, timeout=60): """Loop and wait for volume backup to reach a specified state.""" loop_counter = int(timeout) / 2 while loop_counter > 0: loop_counter -= 1 try: bck = self.client.backups.get(bck_id) if bck.status not in status_list: time.sleep(2) else: break except Exception as e: raise Exception("api:BCKGET #1 Failed : %s" % (e)) else: raise Exception("api:BCKGET #1 timed out") self.print("Backup: %s went to state %s." % (bck_id, bck.status)) return bck.status def _api_tests_undo(self, vol_id, bck_id=None, instance_id=None): """Perform requested tidyup; on best-effort basis""" if instance_id is not None: try: self.novaclient.servers.get(instance_id) self.novaclient.servers.delete(instance_id) except cinderclient.exceptions.NotFound: pass except Exception as e: self.print("Failed to delete test instance: %s" % e) if vol_id is not None: try: # Give the volume one more chance to get into a deleteable # state. self._wait_for_status(vol_id, ['available', 'error']) vol = self.client.volumes.get(vol_id) self.client.volumes.delete(vol) except cinderclient.exceptions.NotFound: pass except Exception as e: self.print("Failed to delete test volume: %s" % e) if bck_id is not None: try: bckup = self.client.backups.get(bck_id) self.client.backups.delete(bckup) except cinderclient.exceptions.NotFound: pass except Exception as e: self.print("Failed to delete test backup: %s" % e) def api_tests_attach(self, vol_id): self.print("Test: API attach volume") # # There's nothing much we can do, except use the first image and # flavor. image = self.novaclient.images.list()[0] flavor = self.novaclient.flavors.list()[0] # But CLI can override: if self.options.image is not None: image = self.options.image self.print("Booting from the image %s" % image) if self.options.flavor is not None: flavor = self.options.flavor self.print("Booting with the flavor %s" % flavor) instance = self.novaclient.servers.create(name="__chkvm__", image=image, flavor=flavor) vm_status = self._wait_for_instance_status(instance.id, ['ACTIVE', 'ERROR']) if vm_status != 'ACTIVE': self._api_tests_undo(None, None, instance.id) raise Exception("api:Instance final status not 'ACTIVE'") try: self.novaclient.volumes.create_server_volume(instance.id, vol_id, None) self._wait_for_status(vol_id, ['in-use', 'error']) self.novaclient.volumes.delete_server_volume(instance.id, vol_id) self._wait_for_status(vol_id, ['available', 'error']) except Exception as e: self._api_tests_undo(vol_id, None, instance.id) raise Exception("api:Instance exception in attach/detach - %s" % e) self.novaclient.servers.delete(instance) def api_tests_backup(self, vol_id, do_restore=True): """Cinder backup API tests: list, create, restore, delete""" self.print("Test: API Backup list") try: test_vol_backup_list = self.client.backups.list() except Exception as e: raise Exception("api:BACKUP list Failed : %s" % (e)) if self.options.verbose: for bck in test_vol_backup_list: self.print("Backup: %s; volume: %s backup status: %s" % (bck.id, bck.volume_id, bck.status)) self.print("Test: API Backup create") try: test_vol_backup = self.client.backups.create( vol_id, name='__chkvolbck__') except Exception as e: raise Exception("api:BACKUP create Failed : %s" % (e)) # volume status will be 'backing-up' for a while # wait for the backup to be done vol_status = self._wait_for_status(vol_id, ['available', 'error']) if vol_status != 'available': self._api_tests_undo(None, test_vol_backup.id) raise Exception("api:BACKUP final status not 'available'") # self.print("Test: API Backup in Swift") # TODO: for added bonus - check the swift container for files if do_restore: self.print("Test: API Backup restore") try: restore = self.client.restores.restore(test_vol_backup.id) restore_vol_id = restore.volume_id except Exception as e: self._api_tests_undo(None, test_vol_backup.id) raise Exception("api:BACKUP restore Failed : %s" % (e)) vol_status = self._wait_for_status(restore_vol_id, ['available', 'error']) if vol_status != 'available': self._api_tests_undo(restore_vol_id, test_vol_backup.id) raise Exception("api:RESTORE final vol status not 'available'") # backup status will be 'restoring' for a while bck_status = self._wait_for_backup_status(test_vol_backup.id, ['available', 'error']) if bck_status != 'available': self._api_tests_undo(restore_vol_id, test_vol_backup.id) raise Exception("api:RESTORE final bck status not 'available'") # Now delete the restored volume self._api_tests_undo(restore_vol_id) self.print("Test: API Backup delete") try: self.client.backups.delete(test_vol_backup) except Exception as e: raise Exception("api:BACKUP delete Failed : %s" % (e)) # TODO: check the backup is actually gone # the backup status goes to 'deleting' def api_tests_common(self, vers): """Basic API tests: list, create, delete""" self.print("Test: API List") try: test_vol_list = self.client.volumes.list() except cinderclient.exceptions.NotFound as c: print("Error: Check your openstack auth url", file=sys.stderr) raise Exception("api:VOLLIST Failed : %s" % (c)) except Exception as e: raise Exception("api:VOLLIST Failed : %s" % (e)) if self.options.verbose: for vol in test_vol_list: self.print("Volume: %s; name: %s status: %s" % (vol.id, (self._name_for_vers(vol, vers)), vol.status)) self.print("Test: API Create - 1GiB volume") try: if vers == '1': test_vol_create = self.client.volumes.create( 1, display_name='__chkvol__') else: test_vol_create = self.client.volumes.create( 1, name='__chkvol__') except Exception as e: raise Exception("api:VOLCREATE Failed : %s" % (e)) if self.options.verbose: self.print("Volume: %s; name: %s status: %s" % (test_vol_create.id, self._name_for_vers(test_vol_create, vers), test_vol_create.status)) # Loop and wait for volume to go active vol_status = self._wait_for_status(test_vol_create.id, ['available', 'error']) if vol_status != 'available': self._api_tests_undo(test_vol_create.id) raise Exception("api:VOLCREATE final status is not 'available'") if self.options.full: try: self.api_tests_backup(test_vol_create.id) self.api_tests_attach(test_vol_create.id) except Exception: self._api_tests_undo(test_vol_create.id) raise self.print("Test: API Delete") try: self.client.volumes.delete(test_vol_create) except cinderclient.exceptions.NotFound as c: raise Exception("api:VOLDELETE Failed : %s" % (c)) except Exception as e: self.print(type(e)) raise Exception("api:VOLDELETE Failed : %s" % (e)) # TODO: check the volume is actually gone if self.options.verbose: try: tmpvol = self.client.volumes.get(test_vol_create.id) self.print("Volume: %s; name: %s status: %s" % (tmpvol.id, self._name_for_vers(tmpvol, vers), tmpvol.status)) except cinderclient.exceptions.NotFound as c: self.print("Volume: %s; name: %s removed...." % (test_vol_create.id, self._name_for_vers(test_vol_create, vers))) except Exception as e: raise Exception("api:VOLGET #2 Failed : %s" % (e)) def main(): if not os.geteuid() == 0: print("You must be 'root' user to run this script", file=sys.stderr) sys.exit(1) create_arguments(argparser) args = argparser.parse_args() test = CinderCheckClient(args) test.run_tests() print("Test completed.") if __name__ == '__main__': main() 0707010015BC07000081A40000000000000000000000015BE091E9000018F3000000FD0000000200000000000000000000004500000000python-cinderlm-0.0.2+git.1541444073.4d3347c/cinderlm/cinder_diag.py#!/usr/bin/env python # # (c) Copyright 2015,2016 Hewlett Packard Enterprise Development LP # (c) Copyright 2017 SUSE LLC # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. # from __future__ import print_function import argparse from cinder_capacity_check import get_capacity import json import os import re import socket from swiftlm.hp_hardware import ssacli from swiftlm.utils.values import Severity import sys import time import yaml PROC_DIR = '/proc' # This python module implements a single cinder service check to # report the number of process of each cinder type running. # In the longer term the process check should be broken out to # constitute one of many newer tests and this file should remain # the driver script for ALL diagnostics. # This name is known by monasca - do NOT change MODULE_SERVICE_NAME = 'block-storage' # The name of metric to be reported MODULE_METRIC_NAME = 'cinderlm.cinder.cinder_services' # Cinder processes for which to perform a count SUBSERVICES = [ "cinder-volume", "cinder-backup", "cinder-api", "cinder-scheduler" ] argparser = argparse.ArgumentParser(usage="Cinder Diagnostics Utility") def create_arguments(parser): """Program arguments.""" client_args = parser.add_argument_group('Cinder diagnostics arguments') client_args.add_argument('-j', '--json', default=False, action='store_true', help='Emit json if True, else emit yaml') client_args.add_argument('--cinder-services', dest='cinder_services', default=False, action='store_true', help='Do a process count of cinder services') client_args.add_argument('--cinder-capacity', dest='cinder_capacity', default=False, action='store_true', help='Do a check on cinder backend capacity') client_args.add_argument('--hpssacli', dest='hpssacli', default=False, action='store_true', help='(deprecated) Check local disk devices. ' 'Use --ssacli instead.') client_args.add_argument('--ssacli', dest='ssacli', default=False, action='store_true', help='Check local disk devices.') def metric(name, value, dimensions, timestamp): """Construct the metric dictionary To list these metrics (for say the last two hours): monasca measurement-list cinderlm.cinder.cinder_services -120 \ --dimensions hostname=standard-ccp-c1-m1-mgmt,component=cinder-volume """ metric = { 'metric': name, 'value': value, 'dimensions': dimensions, 'timestamp': timestamp, } if value > 0: msg = "%s is running" % dimensions['component'] else: msg = "%s is not running" % dimensions['component'] metric['value_meta'] = {'msg': msg} return metric def _check_process(name): found = 0 pid_re = re.compile("^\d+") for sub_directory in [d for d in os.listdir(PROC_DIR) if pid_re.match(d)]: if (os.path.exists(os.path.join(PROC_DIR, sub_directory, "cmdline"))): cmdline = os.path.join(PROC_DIR, sub_directory, "cmdline") try: if name in open(cmdline, "r").read(): found += 1 except IOError: continue return found def check_process(name): return _check_process(name) def check_cinder_processes(): results = [] for subservice in SUBSERVICES: val = check_process(subservice) c = metric(MODULE_METRIC_NAME, val, {'service': MODULE_SERVICE_NAME, 'hostname': socket.gethostname(), 'component': subservice}, time.time()) results.append(c) return results def check_ssacli(): """GET local smart array status Wrap swiftlm ssacli diag to get results """ # Needs root privileges to run results, slots = ssacli.get_smart_array_info() if type(results) != list: # A single metric can be emitted in some cases: # # swiftlm.hp_hardware.ssacli.smart_array failed with: \ # flock: failed to execute ssacli: Permission denied results = [results] for slot in slots: results.extend(ssacli.get_physical_drive_info(slot)) results.extend(ssacli.get_logical_drive_info(slot, cache_check=True)) for result in results: # where possible change the service strings result.name = result.name.replace('swift', 'cinder') for key in result.dimensions.keys(): if key == 'service': result.dimensions[key] = 'block-storage' result.dimensions[key] = result.dimensions[key].replace('swift', 'cinder') # To print individual results do this... # for result in results: # print(repr(result)) return [result.metric() for result in results] def main(): create_arguments(argparser) args = argparser.parse_args() results = [] if args.cinder_services: results = check_cinder_processes() if args.cinder_capacity: results.extend(get_capacity()) if args.hpssacli: results.extend(check_ssacli()) if args.ssacli: results.extend(check_ssacli()) if args.json: print(json.dumps(results, sort_keys=True, indent=4)) else: yaml.add_representer(Severity, Severity.yaml_repr, yaml.SafeDumper) print(yaml.safe_dump(results, allow_unicode=True, default_flow_style=False)) sys.exit(0) if __name__ == '__main__': main() 0707010015BBFE000041ED0000000000000000000000045BE091E900000000000000FD0000000200000000000000000000003E00000000python-cinderlm-0.0.2+git.1541444073.4d3347c/cinderlm/monasca0707010015BC02000081A40000000000000000000000015BE091E900000000000000FD0000000200000000000000000000004A00000000python-cinderlm-0.0.2+git.1541444073.4d3347c/cinderlm/monasca/__init__.py0707010015BBFF000041ED0000000000000000000000025BE091E900000000000000FD0000000200000000000000000000004C00000000python-cinderlm-0.0.2+git.1541444073.4d3347c/cinderlm/monasca/check_plugins0707010015BC01000081A40000000000000000000000015BE091E900000000000000FD0000000200000000000000000000005800000000python-cinderlm-0.0.2+git.1541444073.4d3347c/cinderlm/monasca/check_plugins/__init__.py0707010015BC00000081A40000000000000000000000015BE091E900002A01000000FD0000000200000000000000000000005E00000000python-cinderlm-0.0.2+git.1541444073.4d3347c/cinderlm/monasca/check_plugins/cinderlm_check.py# # (c) Copyright 2015,2016 Hewlett Packard Enterprise Development LP # (c) Copyright 2017-2018 SUSE LLC # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. # # In case you are tempted to import from non-built-in libraries, think twice: # this module will be imported by monasca-agent which must therefore be able # to import any dependent modules. from __future__ import print_function from collections import defaultdict import glob import json from monasca_agent.collector import checks import os import socket import subprocess import threading import time OK = 0 WARN = 1 FAIL = 2 UNKNOWN = 3 # name used for metrics reported directly by this module e.g. when a task # fails or times out. (we need to hard code this name rather than use the # module name because the module name reported by __name__ is dependant on how # monasca-agent imports the module) MODULE_METRIC_NAME = 'cinderlm.cinderlm_check' SERVICE_NAME = 'block-storage' def create_task_failed_metric(task_type, task_name, reason=""): """Generate metric to report that a task has raised an exception.""" return dict( metric=MODULE_METRIC_NAME, dimensions={'type': task_type, 'task': task_name, 'service': SERVICE_NAME, 'hostname': socket.gethostname()}, # value_meta is limited to size 2048, truncate the reason # to 2047 in length if it could contain a large traceback value_meta=dict( msg=('%s task %s execution failed: "%s"' % (task_type.title(), task_name, reason))[:2047]), value=FAIL) def create_timed_out_metric(task_type, task_name): """Generate metric to report that a task has timed out.""" return dict( metric=MODULE_METRIC_NAME, dimensions={'type': task_type, 'task': task_name, 'service': SERVICE_NAME, 'hostname': socket.gethostname()}, value_meta=dict( msg='%s task execution timed out: "%s"' % (task_type.title(), task_name)), value=FAIL) def create_success_metric(task_type, task_name): """Generate metric to report that a task successful.""" return dict( metric=MODULE_METRIC_NAME, dimensions={'type': task_type, 'task': task_name, 'service': SERVICE_NAME, 'hostname': socket.gethostname()}, value_meta=dict( msg='%s task execution succeeded: "%s"' % (task_type.title(), task_name)), value=OK) class CommandRunner(object): def __init__(self, command): self.command = command self.stderr = self.stdout = self.returncode = self.exception = None self.timed_out = False self.process = None def run_with_timeout(self, timeout): thread = threading.Thread(target=self.run_subprocess) thread.start() thread.join(timeout) if thread.is_alive(): self.timed_out = True if self.process: self.process.terminate() def run_subprocess(self): try: self.process = subprocess.Popen( self.command, stdout=subprocess.PIPE, stderr=subprocess.PIPE) self.stdout, self.stderr = self.process.communicate() self.returncode = self.process.returncode except Exception as e: # noqa self.exception = e class CinderLMScan(checks.AgentCheck): # set of check tasks implemented, valid tasks are # 'cinder-services' # 'cinder-capacity' # Tasks added here will be executed by the monasca check process # sequentially in separate processes. We moved the capacity and services # tasks to a cron job to improve the perfomance of monasca check in # response to CINDER-405 TASKS = ( ) # command args to be used for all calls to shell commands COMMAND_ARGS = ['/usr/bin/cinder_diag', '--json'] COMMAND_TIMEOUT = 15.0 SUBCOMMAND_PREFIX = '--' # list of sub-comands each of which is appended to a shell command # with the prefix added DEFAULT_SUBCOMMANDS = TASKS def __init__(self, name, init_config, agent_config, instances=None, logger=None): super(CinderLMScan, self).__init__( name, init_config, agent_config, instances) self.log = logger or self.log def log_summary(self, task_type, summary): task_count = len(summary.get('tasks', [])) if task_count == 1: msg = 'Ran 1 %s task.' % task_type else: msg = 'Ran %d %s tasks.' % (task_count, task_type) # suppress log noise if no tasks were configured logger = self.log.info if task_count else self.log.debug logger(msg) def _run_command_line_task(self, task_name): # we have to call out to a command line command = list(self.COMMAND_ARGS) command.append(self.SUBCOMMAND_PREFIX + task_name) cmd_str = ' '.join(command) runner = CommandRunner(command) try: runner.run_with_timeout(self.COMMAND_TIMEOUT) except Exception as e: # noqa self.log.warn('Command:"%s" failed to run with error:"%s"' % (cmd_str, e)) metrics = create_task_failed_metric('command', task_name, e) else: if runner.exception: self.log.warn('Command:"%s" failed during run with error:"%s"' % (cmd_str, runner.exception)) metrics = create_task_failed_metric('command', task_name, runner.exception) elif runner.timed_out: self.log.warn('Command:"%s" timed out after %ss' % (cmd_str, self.COMMAND_TIMEOUT)) metrics = create_timed_out_metric('command', cmd_str) elif runner.returncode: self.log.warn('Command:"%s" failed with status:%s stderr:%s' % (cmd_str, runner.returncode, runner.stderr)) metrics = create_task_failed_metric('command', task_name, runner.stderr) else: try: metrics = json.loads(runner.stdout) metrics.append(create_success_metric('command', task_name)) except (ValueError, TypeError) as e: self.log.warn('Failed to parse json: %s' % e) metrics = create_task_failed_metric('command', task_name, e) return metrics def _get_metrics(self, task_names, task_runner): reported = [] summary = defaultdict(list) for task_name in task_names: summary['tasks'].append(task_name) metrics = task_runner(task_name) if not isinstance(metrics, list): metrics = [metrics] for metric in metrics: reported.append(metric) return reported, summary def _load_json_file(self, json_file): with open(json_file, 'rb') as f: all_json = json.load(f) return all_json def _get_file_metrics(self, argsfile): reported = [] errors = [] jfile = None for jfile in glob.glob(argsfile): if os.path.isfile(jfile): try: reported.extend(self._load_json_file(jfile)) except Exception as e: errors.extend(['Error: error loading JSON file %s: %s' % (jfile, e)]) continue else: errors.extend(['Error: specified input(%s) is not a file' % jfile]) continue if jfile is None: errors.extend(['Warning: no specified input file(%s) exists' % argsfile]) # emit errors but continue to print json for msg in errors: self.log.error(msg) # Fake out the timestamp with the current timestamp - we are submitting # as if its NOW ts = time.time() for result in reported: result['timestamp'] = ts return reported def _csv_to_list(self, csv): return [f.strip() for f in csv.split(',') if f] def _load_instance_config(self, instance): self.log.debug('instance config %s' % str(instance)) if instance.get('subcommands') is None: self.subcommands = self.DEFAULT_SUBCOMMANDS else: self.subcommands = self._csv_to_list(instance.get('subcommands')) self.log.debug('Using subcommands %s' % str(self.subcommands)) def check(self, instance): self._load_instance_config(instance) # run command line tasks all_metrics, summary = self._get_metrics( self.subcommands, self._run_command_line_task) self.log_summary('command', summary) # gather metrics logged to directory all_metrics.extend( self._get_file_metrics('/var/cache/cinderlm/*.json')) for metric in all_metrics: # apply any instance dimensions that may be configured, # overriding any dimension with same key that check has set. metric['dimensions'] = self._set_dimensions(metric['dimensions'], instance) self.log.debug( 'metric %s %s %s %s' % (metric.get('metric'), metric.get('value'), metric.get('value_meta'), metric.get('dimensions'))) try: self.gauge(**metric) except Exception as e: # noqa self.log.exception('Exception while reporting metric: %s' % e) 0707010015BC03000041ED0000000000000000000000025BE091E900000000000000FD0000000200000000000000000000004D00000000python-cinderlm-0.0.2+git.1541444073.4d3347c/cinderlm/monasca/detect_plugins0707010015BC04000081A40000000000000000000000015BE091E900000000000000FD0000000200000000000000000000005900000000python-cinderlm-0.0.2+git.1541444073.4d3347c/cinderlm/monasca/detect_plugins/__init__.py0707010015BC05000081A40000000000000000000000015BE091E900000877000000FD0000000200000000000000000000006000000000python-cinderlm-0.0.2+git.1541444073.4d3347c/cinderlm/monasca/detect_plugins/cinderlm_detect.py# # (c) Copyright 2015 Hewlett Packard Enterprise Development LP # (c) Copyright 2017 SUSE LLC # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. # # In case you are tempted to import from non-built-in libraries, think twice: # this module will be imported by monasca-agent which must therefore be able # to import any dependent modules. import logging from monasca_setup import agent_config import monasca_setup.detection from monasca_setup.detection.utils import _get_dimensions log = logging.getLogger(__name__) class CinderLMDetect(monasca_setup.detection.ArgsPlugin): """Detect if we will be monitoring Cinder.""" CHECK_NAME = 'cinderlm_check' def __init__(self, template_dir, overwrite=True, args=None): super(CinderLMDetect, self).__init__( template_dir, overwrite, args) def _detect(self): """Run detection, set self.available True if config is detected.""" # (Called during superclass __init__). self.available = True def build_config(self): """Build the config as a Plugins object and return.""" config = agent_config.Plugins() parameters = {'name': self.CHECK_NAME} # set service and component dimensions = _get_dimensions('block-storage', None) if len(dimensions) > 0: parameters['dimensions'] = dimensions config[self.CHECK_NAME] = {'init_config': None, 'instances': [parameters]} log.info("\tEnabling the CinderLMDetect plugin %s" % config) return config def dependencies_installed(self): """Return True if dependencies are installed.""" return True 0707010015BC0F000081A40000000000000000000000015BE091E900000054000000FD0000000200000000000000000000003E00000000python-cinderlm-0.0.2+git.1541444073.4d3347c/requirements.txtpython-cinderclient python-swiftclient python-glanceclient python-novaclient pyyaml 0707010015BC0D000081A40000000000000000000000015BE091E90000072E000000FD0000000200000000000000000000003600000000python-cinderlm-0.0.2+git.1541444073.4d3347c/setup.py#!/usr/bin/env python # # (c) Copyright 2015 Hewlett Packard Enterprise Development LP # (c) Copyright 2017-2018 SUSE LLC # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. # import os import setuptools def read(fname): return open(os.path.join(os.path.dirname(__file__), fname)).read() def requirements(): here = os.path.abspath(os.path.dirname(__name__)) with open(here + '/requirements.txt', 'r') as f: return [y.strip() for y in f.readlines() if y.strip()] reqs = requirements() setuptools.setup( name="cinderlm", version="0.0.3", description="Lifecycle management for cinder", long_description=read('README.md'), author="SUSE LLC", classifiers=[ "Development Status :: 1 - Alpha", "Topic :: Utilities", "License :: OSI Approved :: Apache 2.0", "Operating System :: POSIX :: Linux", "Programming Language :: Python", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 2.6", ], packages=setuptools.find_packages(exclude=['docs', 'etc', 'tests']), install_requires=reqs, entry_points={ 'console_scripts': [ 'cinder_check = cinderlm.cinder_check:main', 'cinder_diag = cinderlm.cinder_diag:main', ], }, ) 0707010015BC0A000081A40000000000000000000000015BE091E900000045000000FD0000000200000000000000000000004300000000python-cinderlm-0.0.2+git.1541444073.4d3347c/test-requirements.txthacking pygments nose nose-exclude coverage nosehtmloutput testtools 0707010015BC0C000081A40000000000000000000000015BE091E900000550000000FD0000000200000000000000000000003500000000python-cinderlm-0.0.2+git.1541444073.4d3347c/tox.ini# Sample tox.ini configuration file. Copy to tox.ini and edit to needs. # More information on the wiki: # https://github.com/sympy/sympy/wiki/Using-Tox # Also see the tox documentation at # http://tox.testrun.org/en/latest/config.html # Note: don't forget to install it: # pip install tox [tox] # Define the environments. By default, py24-py32, jython, pypy. # It's also possible to define a custom environment, like docs. # You must list a defined environment here in order to run it. envlist = py27,py33,py34,pep8 skipdists = True [testenv] # Commands to be executed, it could be anything. The brackets are # important, as they allow us to call bin/test with arguments we # pass on through tox. Applies to all default environments. # You can run multiple commands by putting them on subsequent lines # Note that you should use bin/test and bin/doctest rather than setup.py # test, because the latter does not work with []. usedevelop = True install_command = pip install -U {opts} {packages} setenv = VIRTUAL_ENV={envdir} deps = -r{toxinidir}/requirements.txt -r{toxinidir}/test-requirements.txt commands = nosetests --cover-package=cinderlm --with-xcover tests [testenv:pep8] commands = flake8 -v cinderlm [flake8] ignore = H101 #select = H102,H103,H201,H404,H501,H903 show-source = True exclude = .venv,.tox,.ropeproject,dist,doc,test,*egg 07070100000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000B00000000TRAILER!!!123 blocks