remove Cylc 7 compatibility mode by oliver-sanders · Pull Request #7325 · cylc/cylc-flow

oliver-sanders · 2026-06-03T10:37:34Z

A set of changes to remove Cylc 7 compatibility mode, issues:

Sibling PRs:

Summary:

remove cylc 7 compatibility mode:
- Closes drop compatibility mode #6849
- Remove Cylc 7 compatibility mode.
- Remove support for validating, (re)installing or (re)starting suite.rc workflows.
- Keep support for CLI commands against workflows running under older versions of Cylc 8.
- Remove legacy functional tests.
- Add integration tests (to ensure suite.rc workflows cannot be started).
setup: unpin Jinja2:
- Closes jinja2: bump version #6850
- Remove the upper pin on Jinja2 allowing the use of newer versions.
suite.rc: schedule removal of remaining support
- Cylc 8 is still suite.rc "aware" for now, it's needed for GUI/Tui/CLI.
- Following the logic laid out in review BACK COMPAT code #7274, this support would be removed in Cylc 8.10.

Check List

I have read CONTRIBUTING.md and added my name as a Code Contributor.
Contains logically grouped changes (else tidy your branch by rebase).
Does not contain off-topic changes - off topic changes:
- jinja2: back support removed interfaces (requested)
- config: remove redundant parser argument (requested in review)
- scan: add test for get_workflow_title (shortfall spotted in review to ensure back-compat not broken)
- parser arguments standardised (unrelated, but a pain for testing)
Applied any dependency changes to both setup.cfg (and conda-environment.yml if present).
Tests are included (or explain why tests are not needed).
Changelog entry included if this is a change that can affect users
Cylc-Doc pull request opened if required at remove documentation for Cylc 7 compatibility mode cylc-doc#939.
If this is a bug fix, PR should be raised against the relevant ?.?.x branch.

* See cylc/cylc-flow#7325

oliver-sanders · 2026-06-11T12:30:45Z

+    asyncio.run(run(options, workflow_id))


 async def run(
-    parser: COP, options: 'Values', workflow_id: str
+    options: 'Values', workflow_id: str


Off-topic change: A couple of scripts were passing through the unused parser argument which was creating hassle for tests. These have been removed so all commands have the same basic interface.

* Support for running workflows defined by `suite.rc` files was removed in Cylc 8.7 (cylc#7325) * However, Cylc 8 remains `suite.rc` "aware" in order to allow the GUI, Tui and CLI to list and operate on workflows running under older versions. * A version has been set for the removal of this support in line with cylc#7274.

hjoliver

Looks good, nice tidy up.

* remove documentation for Cylc 7 compatibility mode * See cylc/cylc-flow#7325 * user guide: add section on cylc versions and compatibility * dictionary

* Support for running workflows defined by `suite.rc` files was removed in Cylc 8.7 (cylc#7325) * However, Cylc 8 remains `suite.rc` "aware" in order to allow the GUI, Tui and CLI to list and operate on workflows running under older versions. * A version has been set for the removal of this support in line with cylc#7274.

oliver-sanders · 2026-07-09T15:38:26Z

Rebased, deconflicted and addressed feedback.

* Support for running workflows defined by `suite.rc` files was removed in Cylc 8.7 (cylc#7325) * However, Cylc 8 remains `suite.rc` "aware" in order to allow the GUI, Tui and CLI to list and operate on workflows running under older versions. * A version has been set for the removal of this support in line with cylc#7274.

oliver-sanders · 2026-07-10T09:14:06Z

Have now added: 9be3f5d

Back support Jinja2 interfaces deprecated at 3.0 and removed at 3.1.
Jinja2 renamed a bunch of functions and deprecated their old names.
Our users didn't spot this as Python warnings are not visible to them.
As of Jinja2: catch and log warnings #7365 (8.6.6) Python warnings
originating from Jinja2 will be turned into Cylc warnings.
To provide a migration window, the old function names will be supported
in Cylc 8.7.x and removed in 8.8.0.

* Support for running workflows defined by `suite.rc` files was removed in Cylc 8.7 (cylc#7325) * However, Cylc 8 remains `suite.rc` "aware" in order to allow the GUI, Tui and CLI to list and operate on workflows running under older versions. * A version has been set for the removal of this support in line with cylc#7274.

hjoliver

Reapproving post @MetRonnie 's much more thorough review! (the changes look good)

MetRonnie · 2026-07-15T11:52:24Z

-                    continue
-                base_point = point
-                break
+            base_point = min(self.get_tasks_by_point(), default=None)


The keys of self.get_tasks_by_point() are the same as self.active_tasks but with some processing of the values that is not needed here.

Suggested change

base_point = min(self.get_tasks_by_point(), default=None)

base_point = min(self.active_tasks.keys(), default=None)

(Off topic), but yes, this is quite inefficient, I'm not sure how it got like this, will change!

Note, this appears to be the only use of get_tasks_by_point!

Note #7400 removes the call to get_tasks_by_point from the Cylc 8 branch.

MetRonnie · 2026-07-15T12:14:29Z

+    Jinja2 warnings and turned them into Cylc warnings to ensure they cannot be
+    missed.
+
+    To give users a chance to take action, this patch restores the old


Unless this was meant to be exhaustive,

Suggested change

To give users a chance to take action, this patch restores the old

To give users a chance to take action, this patch restores some of the old

This comment is scoped to renamed interfaces (not removed or modified interfaces):

From this line above:

Jinja2 renamed a bunch of interfaces. Warnings were added in version 3.0.0,

It should restore all of the renamed interfaces.

I would add a note that this does not cover all the removed deprecations: https://jinja.palletsprojects.com/en/stable/changes/#version-3-1-0

Will clarify thus:

Suggested change

To give users a chance to take action, this patch restores the old

To give users a chance to take action, this patch restores the renamed

This comment does not pertain to the removed deprecations, only the renamed ones.

* Support for running workflows defined by `suite.rc` files was removed in Cylc 8.7 (cylc#7325) * However, Cylc 8 remains `suite.rc` "aware" in order to allow the GUI, Tui and CLI to list and operate on workflows running under older versions. * A version has been set for the removal of this support in line with cylc#7274.

oliver-sanders · 2026-07-17T10:18:57Z

@MetRonnie, please the recent force-push which address feedback above:

Remove the obsolete TaskPool.get_tasks_by_point interface.
Misc minor stuff.
Add a test for get_workflow_title which also ensures this interface continues to work with suite.rc files.
Refactors whole workflow config file detection stuff, the previous approach was hard to follow, but the motive was restoring support for flow.cylc -> suite.rc sylinks which we apparently supported before (support I accidentally removed earlier).

@hjoliver, you may want to look over the last four commits:

jinja2: back support removed interfaces:
- Context here https://matrix.to/#/!gMuCeOJBhYqgWDGwnR:matrix.org/$17835256323042zibit:matrix.org?via=matrix.org
- Icky back-compat support to be removed ASAP in 8.8
refactor workflow detection logic:
- Clean up the code which detects suite.rc / flow.cylc files and determines what is and isn't a valid workflow config.
scan: add test for get_workflow_title:
- (Unimportant, a Cylc 7 gcylc thing).
task_pool: remove obsolete get-tasks_by_point interface:
- Small unrelated efficiency improvement spotted during review.

MetRonnie

(Got a test failure)

MetRonnie · 2026-07-17T12:11:30Z

+    # flow.cylc present
+    (tmp_path / WorkflowFiles.SUITE_RC).unlink()
+    with pytest.raises(WorkflowConfigError):
+        await validate(ValidateOptions(), str(tmp_path))


This should not raise an error, as flow.cylc only should be ok? (Currently the error is because the file is empty so not a valid config)

The previous two examples raise WorkflowFilesError, this one raises WorkflowConfigError which demonstrates the flow.cylc file is being accepted.

A little confusing admirably, I've added a match pattern and a comment to clarify.

* Closes cylc#6849 * Remove Cylc 7 compatibility mode. * Remove support for validating, (re)installing or (re)starting `suite.rc` workflows. * Keep support for CLI commands against workflows running under older versions of Cylc 8. * Remove legacy functional tests. * Add integration tests (to ensure `suite.rc` workflows cannot be started). * Remove redundant Parsec `silent` option.

* Closes cylc#6850 * Remove the upper pin on Jinja2 allowing the use of newer versions.

* The flags module contains global variables, generally considered an anti-pattern. * We're down to the last flag, add a note to discourage the addition of any new ones!

* Support for running workflows defined by `suite.rc` files was removed in Cylc 8.7 (cylc#7325) * However, Cylc 8 remains `suite.rc` "aware" in order to allow the GUI, Tui and CLI to list and operate on workflows running under older versions. * A version has been set for the removal of this support in line with cylc#7274.

* Back support Jinja2 interfaces deprecated at 3.0 and removed at 3.1. * Jinja2 renamed a bunch of functions and deprecated their old names. * Our users didn't spot this as Python warnings are not visible to them. * As of cylc#7365 (8.6.6) Python warnings originating from Jinja2 will be turned into Cylc warnings. * To provide a migration window, the old function names will be supported in Cylc 8.7.x and removed in 8.8.0.

* Reinstate support for `flow.cylc -> suite.rc` symlinks (which we've supported to date with no warnings).

* Test will ensure `suit.rc` files remain supported post removal of Cylc 7 compatibility mode until purposefully withdrawn in a later version.

oliver-sanders · 2026-07-28T15:29:16Z

(ISC004 lint failure to be fixed on 8.6.x)

oliver-sanders · 2026-07-28T15:36:05Z

Latest changes.

Outstanding feedback addressed.
Warning mechanism introduced to ensure those leaning on back-supported interfaces get warned appropriately.
Functional tests fixed (log message format change).

hjoliver · 2026-07-29T00:07:47Z

@hjoliver, you may want to look over the last four commits: ...

LGTM

hjoliver · 2026-07-29T00:09:08Z

N.B. I had also noticed get_tasks_by_point in other work: #7325 (comment)

oliver-sanders added this to the 8.7.0 milestone Jun 3, 2026

oliver-sanders self-assigned this Jun 3, 2026

oliver-sanders changed the title ~~Compat~~ remove Cylc 7 compatibility mode Jun 3, 2026

oliver-sanders mentioned this pull request Jun 3, 2026

remove documentation for Cylc 7 compatibility mode cylc/cylc-doc#939

Merged

3 tasks

oliver-sanders added a commit to oliver-sanders/cylc-doc that referenced this pull request Jun 3, 2026

remove documentation for Cylc 7 compatibility mode

727b784

* See cylc/cylc-flow#7325

oliver-sanders force-pushed the compat branch from 1e9e781 to 8b50814 Compare June 3, 2026 12:19

oliver-sanders mentioned this pull request Jun 11, 2026

remove Cylc 7 compatibility mode cylc/cylc-rose#429

Open

10 tasks

oliver-sanders marked this pull request as ready for review June 11, 2026 12:28

oliver-sanders commented Jun 11, 2026

View reviewed changes

oliver-sanders requested a review from hjoliver June 18, 2026 13:40

hjoliver reviewed Jun 25, 2026

View reviewed changes

Comment thread cylc/flow/cfgspec/workflow.py Outdated

hjoliver approved these changes Jun 25, 2026

View reviewed changes

oliver-sanders requested a review from MetRonnie June 25, 2026 09:14

MetRonnie reviewed Jun 26, 2026

View reviewed changes

MetRonnie reviewed Jun 29, 2026

View reviewed changes

MetRonnie mentioned this pull request Jul 6, 2026

Jinja2: catch and log warnings #7365

Merged

8 tasks

oliver-sanders force-pushed the compat branch from 72669b7 to 423bd4a Compare July 9, 2026 15:37

oliver-sanders force-pushed the compat branch from 423bd4a to c7493d2 Compare July 10, 2026 09:13

This was referenced Jul 10, 2026

8.7: followup to 7365 #7368

Open

8.6: followup to 7365 #7376

Open

oliver-sanders force-pushed the compat branch from c7493d2 to 9be3f5d Compare July 10, 2026 09:47

oliver-sanders requested a review from MetRonnie July 10, 2026 14:46

hjoliver approved these changes Jul 13, 2026

View reviewed changes

MetRonnie reviewed Jul 15, 2026

View reviewed changes

oliver-sanders force-pushed the compat branch from 9be3f5d to 5680d31 Compare July 17, 2026 10:07

oliver-sanders requested a review from MetRonnie July 17, 2026 10:19

MetRonnie reviewed Jul 17, 2026

View reviewed changes

oliver-sanders added 9 commits July 23, 2026 13:32

setup: unpin Jinja2

fe01ce5

* Closes cylc#6850 * Remove the upper pin on Jinja2 allowing the use of newer versions.

config: remove redundant parser argument

c8c3a92

flags: deprecate module

f9e6ac0

* The flags module contains global variables, generally considered an anti-pattern. * We're down to the last flag, add a note to discourage the addition of any new ones!

refactor workflow detection logic:

d6ba1a1

* Reinstate support for `flow.cylc -> suite.rc` symlinks (which we've supported to date with no warnings).

scan: add test for get_workflow_title

8b90f2e

* Test will ensure `suit.rc` files remain supported post removal of Cylc 7 compatibility mode until purposefully withdrawn in a later version.

task_pool: remove obsolete get_tasks_by_point interface

c043031

oliver-sanders force-pushed the compat branch from 5680d31 to c043031 Compare July 28, 2026 15:26

	base_point = min(self.get_tasks_by_point(), default=None)
	base_point = min(self.active_tasks.keys(), default=None)

	To give users a chance to take action, this patch restores the old
	To give users a chance to take action, this patch restores some of the old

Uh oh!

Conversation

oliver-sanders commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

oliver-sanders Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

hjoliver left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

oliver-sanders commented Jul 9, 2026

Uh oh!

oliver-sanders commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hjoliver left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

MetRonnie Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

oliver-sanders Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hjoliver Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

MetRonnie Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

oliver-sanders Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MetRonnie Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

oliver-sanders Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

oliver-sanders commented Jul 17, 2026

Uh oh!

MetRonnie left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

MetRonnie Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

oliver-sanders Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

oliver-sanders commented Jul 28, 2026

Uh oh!

oliver-sanders commented Jun 3, 2026 •

edited

Loading

oliver-sanders Jun 11, 2026 •

edited

Loading

oliver-sanders commented Jul 10, 2026 •

edited

Loading

oliver-sanders Jul 15, 2026 •

edited

Loading

oliver-sanders Jul 15, 2026 •

edited

Loading

oliver-sanders Jul 17, 2026 •

edited

Loading