MAPI - Log Resources¶
New in version 0.9.3.
This class allows to request internal logs from HCP, which will then prepare a zip-file to be downloaded later on.
Note
To be able to use this class, HCP needs to run at least version 7.2.
The usage pattern is this:
- prepare()
- check the status() until readyForStreaming is True, then
- download() the logs
- save the zip’ed logs or process them, what ever is needed.
- close() the underlying Connection() object
Classes¶
Logs¶
-
class
hcpsdk.mapi.Logs(target, debuglevel=0)[source]¶ Access to HCP internal logfiles (ACCESS, SYSTEM, SERVICE, APPLICATION)
Parameters: - target – an hcpsdk.Target object
- debuglevel – 0..9 (used in http.client)
Raises: - hcpsdk.HcpsdkPortError in case target is
initialized with an incorrect port for use by this class.
Class constants:
Log types:
-
L_ACCESS¶ The log files http_gateway_request.log.x, mapi_gateway_request.log.x and admin_request.log.x contain all HTTP access messages that includes all of the HTTP based (SMC, TMC, MAPI, MQE, REST, HS3, HSwift) gateway logs. MAPI and Admin will contain all the system level access information while http gateway contains TMC and namespace access info.
-
L_SYSTEM¶ This is a snapshot of the running system when the logs were captured. This includes the messages that are generated by HCP OS, the firewall and routing rules, the fiber channel info and many others and are always important as a first step for support to go over.
-
L_SERVICE¶ These are logs generated when performing any of the service procedures such as when installing HCP software, adding a node, adding a LUN, node recovery etc.
-
L_APPLICATION¶ These are logs from all the HCP specific services such as the JVM, the volume manager, storman etc.
-
L_ALL¶ A list containing all other L_* log types.
Class attributes
-
suggestedfilename¶ When a download has been started, this attribute holds the filename suggested by HCP.
Class methodes:
-
mark(message)[source]¶ Mark HCPs internal log with a message.
Parameters: message – the message to be logged Raises: LogsError
-
prepare(startdate=None, enddate=None, snodes=[])[source]¶ Command HCP to prepare logs from startdate to enddate for later download.
Parameters: - startdate – 1st day to collect (as a datetime.date object)
- enddate – last day to collect (as a datetime.date object)
- snodes – list of S-series nodes to collect from
Returns: a tuple of datetime.date(startdate), datetime.date(enddate) and str(prepared XML)
Raises: ValueError arguments are invalid or one of the LogsError if an operation failed
-
status()[source]¶ Query HCP for the status of the request log download.
Returns: a collection.OrderedDict: { readyForStreaming: bool, streamingInProgress: bool, started: bool, error: bool, content: list # one or more of: L_ACCESS, L_SYSTEM, # L_SERVICE, L_APPLICATION) }
Raises: re-raises whatever is raised below
-
download(hdl=None, nodes=[], snodes=[], logs=[], progresshook=None, hidden=True)[source]¶ Download the requested logs.
Parameters: - hdl – a file (or file-like) handle open for binary read/write or None, in which case a temporary file will be created
- nodes – list of node-IDs (int), all if empty
- snodes – list of S-node names (str), none if empty
- logs – list of logs (L_*), all if empty
- progresshook – a function taking a single argument (the # of bytes received) that will be called after each chunk of bytes downloaded
- hidden – the temporary file created will be hidden if possible (see tempfile.TemporaryFile())
Returns: a 2-tuple: the file handle holding the received logs, positioned at byte 0 and the filename suggested by HCP
Raises: LogsError or LogsNotReadyError
Exceptions¶
-
exception
hcpsdk.mapi.LogsError(reason)[source]¶ Base Exception used by the hcpsdk.mapi.Logs() class.
Parameters: reason – An error description
Sample Code¶
The following example code
creates a simple command processor that allows to prepare and download logs
from HCP.
# -*- coding: utf-8 -*- # The MIT License (MIT) # # Copyright (c) 2014-2023 Thorsten Simons (sw@snomis.eu) # # Permission is hereby granted, free of charge, to any person obtaining a copy of # this software and associated documentation files (the "Software"), to deal in # the Software without restriction, including without limitation the rights to # use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of # the Software, and to permit persons to whom the Software is furnished to do so, # subject to the following conditions: # # The above copyright notice and this permission notice shall be included in all # copies or substantial portions of the Software. # # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS # FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR # COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER # IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN # CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. import sys from pprint import pprint import logging import cmd from datetime import date, timedelta import hcpsdk USR = 'logmon' PWD = 'logmon01' TGT = 'admin.hcp72.archivas.com' PORT = hcpsdk.P_MAPI class LogsShell(cmd.Cmd): intro = 'HCP Log Download Shell. Type help or ? to list commands.\n' prompt = '==> ' def preloop(self): self.nodes = [] self.snodes = [] self.logs = [] self.start = date.today() - timedelta(days=7) self.end = date.today() def do_what(self, arg): 'what - show what is selected for download' print('start date: {}'.format(self.start.strftime('%Y/%m/%d'))) print('end date: {}'.format(self.end.strftime('%Y/%m/%d'))) print('selected nodes: {}'.format(' '.join(self.nodes) or \ ' '.join([x.split('.')[3] for x in l.target.addresses]))) print('selected snodes: {}'.format(' '.join(self.snodes) or 'none')) print('selected logs: {}'.format(' '.join(self.logs) or \ ' '.join(hcpsdk.mapi.Logs.L_ALL))) def do_status(self, arg): 'status - query HCP for the log preparation status' log.debug('do_status() called') try: pprint(l.status()) except Exception as e: print(e) def do_prepare(self, arg): 'prepare - trigger HCP to prepare the logs for later download' try: l.prepare(snodes=self.snodes, startdate=self.start, enddate=self.end) except Exception as e: print('prepare failed: {}'.format(e)) else: print('preparing for nodes: {}\n' ' snodes: {}\n' ' date range: {} - {}' .format(' '.join(self.nodes) or 'all', ' '.join(self.snodes) or 'none', self.start.strftime('%Y/%m/%d'), self.end.strftime('%Y/%m/%d'))) def do_nodes(self, arg): 'nodes [node_id,]* - select the nodes to download logs from\n'\ ' (no argument selects all nodes)' if arg: self.nodes = [] n = [x.split('.')[3] for x in l.target.addresses] for x in arg.split(','): if x in n: self.nodes.append(x) else: print('invalid: {}'.format(x)) else: self.nodes = [] def do_snodes(self, arg): 'snodes [snode_name,]* - select the S-nodes to download logs from\n'\ ' (no argument disables S-Node log download)' if arg: self.snodes = arg.split(',') else: self.snodes = [] def do_logs(self, arg): 'logs [ACCESS,|SYSTEM,|SERVICE,|APPLICATION]* - select log types\n'\ ' nothing selects all' if arg: self.logs = [] for x in [y.upper() for y in arg.split(',')]: if x in hcpsdk.mapi.Logs.L_ALL: self.logs.append(x) else: print('invalid: {}'.format(x)) else: self.logs = [] def do_start(self, arg): 'start YYYY/MM/DD - select start date (default is a week ago)' try: d = arg.split('/') self.start = date(int(d[0]),int(d[1]),int(d[2])) except Exception as e: print('invalid input - YYYY/MM/DD required...') def do_end(self, arg): 'end YYYY/MM/DD - select end date (default is today)' try: d = arg.split('/') self.end = date(int(d[0]),int(d[1]),int(d[2])) except Exception as e: print('invalid input - YYYY/MM/DD required...') def do_download(self, filename, ): 'download <filename> - donwload logs and store into file <filename>' if not filename: print('Error: filename missing!') return print('downloading for nodes: {}\n' ' snodes: {}\n' ' logs: {}\n' ' date range: {} - {}\n' .format(' '.join(self.nodes) or \ ' '.join([x.split('.')[3] for x in l.target.addresses]), ' '.join(self.snodes) or 'none', ' '.join(self.logs) or \ ' '.join(hcpsdk.mapi.Logs.L_ALL,), self.start.strftime('%Y/%m/%d'), self.end.strftime('%Y/%m/%d'))) try: with open(filename, 'w+b') as outhdl: hdl, n = l.download(hdl=outhdl, nodes=self.nodes, snodes=self.snodes, logs=self.logs, progresshook=showdownloadprogress) print('\nHCP suggested filename: {}'.format(n)) except Exception as e: print('Error: {}'.format(e)) else: print('download finished') def do_cancel(self, arg): 'cancel - abort a log preparation' try: l.cancel() except Exception as e: print('cancel failed: {}'.format(e)) else: print('cancel done') def do_mark(self, arg): 'mark - mark HCPs log with a message' try: l.mark(arg) except Exception as e: print('mark failed: {}'.format(e)) else: print('log marked') def do_quit(self, arg): 'quit - exit the HCP Logs Shell' self.close() print('Bye...') return True def do_debug(self, arg): 'debug - Toggle debug output' if log.getEffectiveLevel() != logging.DEBUG: log.setLevel(logging.DEBUG) log.debug('debug enabled') print('debug enabled') else: log.setLevel(logging.CRITICAL) print('debug disabled') def emptyline(self): 'run the status command if no command is given' return self.onecmd('status') def close(self): 'close the underlying *hcpsdk.Logs()* object' l.close() def showdownloadprogress(nBytes): ''' Print a simple progress meter ''' sz = ["B", "kB", "MB", "GB", "TB", "PB", "EB", "ZB", "YB"] i = 0 while nBytes > 1023: nBytes = nBytes / 1024 i = i + 1 print("\rreceived: {0:.2f} {1:}".format(nBytes, sz[i]), end='') if __name__ == '__main__': auth = hcpsdk.NativeAuthorization(USR,PWD) try: t = hcpsdk.Target(TGT, auth, port=PORT) except hcpsdk.ips.IpsError as e: sys.exit('Can\'t resolve "{}"\n\t==> {}'.format(TGT, e)) l = hcpsdk.mapi.Logs(t, debuglevel=0) # create console handler with a higher log level sh = logging.StreamHandler(sys.stderr) fh = logging.Formatter("[%(levelname)-8s]: %(message)s") sh.setFormatter(fh) log = logging.getLogger() log.addHandler(sh) # this makes the logger silent, until *debug* has activated log.setLevel(logging.CRITICAL) LogsShell().cmdloop()
Sample Output¶
HCP Log Download Shell. Type help or ? to list commands. ==> logs access ==> snodes snode70 ==> start 2015/09/23 ==> what start date: 2015/09/23 end date: 2015/09/25 selected nodes: 176 selected snodes: snode70 selected logs: ACCESS ==> prepare preparing for nodes: all snodes: snode70 date range: 2015/09/23 - 2015/09/25 ==> status {'readyForStreaming': False, 'streamingInProgress': False, 'error': False, 'started': True, 'content': ['ACCESS', 'SYSTEM', 'SERVICE', 'APPLICATION']} ==> status {'readyForStreaming': True, 'streamingInProgress': False, 'error': False, 'started': True, 'content': ['ACCESS', 'SYSTEM', 'SERVICE', 'APPLICATION']} ==> download logs.zip downloading for nodes: 176 snodes: snode70 logs: ACCESS date range: 2015/09/23 - 2015/09/25 received: 11.41 MB download finished ==> cancel cancel done ==> quit Bye...
