Fix logging rotation

[MetRonnie]

Supersedes and contains two of the bug fixes from #789

Closes #742, closes #743

Fixed logging issues:

• Separated log files for UI servers run by cylc gui and JupyterHub - the latter now uses hubapp-log instead of log.
• Log file rotation was not always working correctly. Log file names now rotate as log.1, log.2 etc. (or hubapp-log.1, hubapp-log.2 etc.).

Check List

• I have read CONTRIBUTING.md and added my name as a Code Contributor.
• Contains logically grouped changes (else tidy your branch by rebase).
• Does not contain off-topic changes (use other PRs for other changes).
• Tests are included
• Changelog entry included
• No docs needed
• If this is a bug fix, PR should be raised against the relevant ?.?.x branch.

[oliver-sanders]

This generally looks good. The standalone and multi-user files are now neatly separated and the rollover is working nicely - no more blank files or other nonsense.

I'm happy with the removal of the symlink, getting it to rename the file instead is perfectly reasonable.

I spotted one issue, it will fall over with traceback if the ~/.cylc/uiserver/log directory doesn't exist when the server is started. We should mkdir -p that at __init__ time.

This does change the filenames to remove the .log extension which is unfortunate as this naming is now inconsistent with other Cylc logs. I'm guessing it changed because that's the logging handler default, but I think we can override the rotation naming via the rotation_filename method or namer attr?

I think as it stands, this will leave any pre-existing <x>-uiserver.log files kicking around (as they are now out of the rotation), which is annoying. I think we could solve this by restoring the original file naming (which would pull the old files back into the rotation), or potentially by shimming something into the rollover code to wipe out files with the previous name (if there's a desire to rename the file for other reasons).

[oliver-sanders]

This is a little icky, but I'm guessing we have no access to application lifecycle information (or even the application itself) at this point so needs must.

It might be clearer to implement this as a global variable as it otherwise looks like the pattern of setting variable default on the class to be shadowed on the application.

[MetRonnie]

This does change the filenames to remove the .log extension which is unfortunate as this naming is now inconsistent with other Cylc logs. I'm guessing it changed because that's the logging handler default, but I think we can override the rotation naming via the rotation_filename method or namer attr?

The *.1, *.2 filename pattern is hard-coded into the RotatingFileHandler unfortunately, so it is not possible to alter without re-implementing the doRollover() method

[MetRonnie]

I spotted one issue, it will fall over with traceback if the ~/.cylc/uiserver/log directory doesn't exist when the server is started. We should mkdir -p that at __init__ time.

Done

[oliver-sanders]

The *.1, *.2 filename pattern is hard-coded into the RotatingFileHandler unfortunately, so it is not possible to alter without re-implementing the doRollover() method

This seems to do the trick:

import logging
import logging.handlers
from pathlib import Path
from time import time


class MyRotatingFileHandler(logging.handlers.RotatingFileHandler):
    def rotation_filename(self, default_name):
        """
        Modify the filename of a log file when rotating.

        This is provided so that a custom filename can be provided.

        The default implementation calls the 'namer' attribute of the
        handler, if it's callable, passing the default name to
        it. If the attribute isn't callable (the default is None), the name
        is returned unchanged.

        :param default_name: The default name for the log file.
        """
        name, number = default_name.rsplit(".", 1)
        path = Path(name)
        return str(path.parent / f"{number}-{path.name}")


LOG = logging.getLogger("a")

LOG.addHandler(MyRotatingFileHandler("logfile.log", maxBytes=100, backupCount=2))
LOG.setLevel(logging.INFO)

LOG.info(f"Hello world {time()} ...........................................")

But we do also have cylc.flow.loggingutil.RotatingFileHandler which it might make sense to use (not sure)?