fix(colab): Add linter for malformed lists in generated notebooks - #3947
fix(colab): Add linter for malformed lists in generated notebooks#3947ribhuji wants to merge 2 commits into
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/tutorials/3947
Note: Links to docs will display an error until the docs builds have been completed. This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
@albanD, could you please approve the workflow run for this draft PR? The current commit intentionally contains only the new Lintrunner check. The job is expected to fail with 14 diagnostics across the 10 tutorials documented in #3946, which will validate that the check detects the existing issues. Once that run is recorded, I’ll push a second commit fixing those violations. This will preserve both the detection run and the final passing run in the PR history. Thank you! |
|
Hey @albanD, bumping this up! |
|
Sorry, I don't really have a lot of time to review things here. What is this one about? |
|
14 Colabs created from the tutorials have indentation problems, adding a Lintrunner check for that. |
|
Why is it not fixing these notebooks? |
|
I don't just want to fix these issues. But I also want to add a check that stops these issues from occurring in any new tutorials as well. If we run the workflow now, the workflow job fails with 14 diagnostics across the 10 tutorials documented in #3946, which will validate that the Lintrunner detects the existing issues, and there is value in having the Lintrunner check being added to this repo. After that I'll push a second commit to this PR fixing these issues, the lintrunner will pass then. This will preserve both the detection run and the final passing run in the PR history. |
1. USB Semi-Supervised LearningTutorial: https://docs.pytorch.org/tutorials/advanced/usb_semisup_learn.html Fixed content: the list of functions imported from semilearn in the Use USB to Train FreeMatch/SoftMatch on CIFAR-10 with only 40 labels section. Colab - https://colab.research.google.com/drive/1pLl7Z1-vo22Z_d9wKxoBsPM6zv9pnpZz?usp=sharing 
2. Automatic Differentiation with torch.autogradTutorial: https://docs.pytorch.org/tutorials/beginner/basics/autogradqs_tutorial.html Fixed content:
Colab - https://colab.research.google.com/drive/106DTdB7xXC7ICHb4kv4cvY9GvPFooH3n?usp=sharing 
3. Datasets & DataLoadersTutorial: https://docs.pytorch.org/tutorials/beginner/basics/data_tutorial.html Fixed content: the FashionMNIST parameter list in the Loading a Dataset section. Colab - https://colab.research.google.com/drive/1Ryu6JY2FgQJqdqwzn5UPSuqz5RoPi2mY?usp=sharing 
4. Neural NetworksTutorial: https://docs.pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html Fixed content: the lists under Recap, At this point, we covered, and Still Left. Colab - https://colab.research.google.com/drive/1kOz58dE3EyV2BzFO_0bJZeh9-ZQ3S2VC?usp=sharing 
5. What is torch.nn really?Tutorial: https://docs.pytorch.org/tutorials/beginner/nn_tutorial.html Fixed content: the list of assumptions in the Wrapping DataLoader section. Colab - https://colab.research.google.com/drive/15YAO3JGZueMIJRPydokTHfpv8gNa0nDX?usp=sharing 
6. Optimizer Step in BackwardTutorial: https://docs.pytorch.org/tutorials/intermediate/optimizer_step_in_backward_tutorial.html Fixed content: the numbered list following Several major observations. Colab - https://colab.research.google.com/drive/1ZYB3KAJUllybwpv6XoYQgvi_DI1j-zDQ?usp=sharing 
7. PyTorch ProfilerTutorial: https://docs.pytorch.org/tutorials/recipes/recipes/profiler_recipe.html Fixed content: the nested activity types under the activities profiler parameter in the Using profiler to analyze execution time section. Colab - https://colab.research.google.com/drive/153edt0JVmGtAkl-C4_Mtbq8xFRlRwd4h?usp=sharing 
8. Timer Quick StartTutorial: https://docs.pytorch.org/tutorials/recipes/recipes/timer_quick_start.html Fixed content: the Contents list near the beginning of the tutorial. Colab - https://colab.research.google.com/drive/1kjvktsPU5fUun1WURCT5nrOfCaAQ7Zdm?usp=sharing 
9. Changing the Compilation StanceTutorial: https://docs.pytorch.org/tutorials/recipes/torch_compiler_set_stance_tutorial.html Fixed content: the list following Other stances include. Colab - https://colab.research.google.com/drive/1Qz2BziA-WoU5A511VkPTSfOnFKI-28zE?usp=sharing 
10. GPU Direct StorageTutorial: https://docs.pytorch.org/tutorials/unstable/gpu_direct_storage.html Fixed content: the list following The steps involved in the process are as follows. Colab - https://colab.research.google.com/drive/1XZDmHG-fy7LhldeIFwIODijuT9fKRdrt?usp=sharing 
Built all 10 with - |
2 participants
Fixes #3946
Description
Some list structures in Sphinx-Gallery tutorial sources render correctly in the HTML documentation but become malformed after conversion to Markdown in the generated Google Colab notebooks.
This PR adds a repository-local
TUTORIAL_MARKUPLintrunner check that detects the two source patterns responsible for the problems documented in #3946:The checker only examines Sphinx-Gallery narrative content from module docstrings and gallery comment blocks. It accounts for valid nested lists, reStructuredText directives, and list tables to avoid reporting those as violations.
Validation strategy
This draft is being built in two commits so the Lintrunner behavior can be verified directly in GitHub Actions:
The expected CI result is 14 diagnostics across exactly the 10 tutorials documented in [BUG] - Incorrect list rendering in Google Colab tutorials #3946, with no additional tutorial files reported.
The subsequent CI run should pass with no
TUTORIAL_MARKUPdiagnostics.Local validation
python3 -m unittest discover -s tools/linter/tests -p 'test_*.py' lintrunner --all-files --take TUTORIAL_MARKUPThe unit test suite currently passes all eight tests. The complete linter scan reports 14 diagnostics across the 10 affected tutorial files and reports no other tutorial files.
Checklist