Skip to content

[API Compatibility] Change compatibility apis to ChangePrefixMatcher, inject paddle.enable_compat() -part#895

Open
Manfredss wants to merge 19 commits into
PaddlePaddle:masterfrom
Manfredss:patch_compat
Open

[API Compatibility] Change compatibility apis to ChangePrefixMatcher, inject paddle.enable_compat() -part#895
Manfredss wants to merge 19 commits into
PaddlePaddle:masterfrom
Manfredss:patch_compat

Conversation

@Manfredss

Copy link
Copy Markdown
Collaborator

PR Docs

For the 23 torch.* APIs that are aligned through paddle.compat.*, switch the
converter to the "prefix-only + enable_compat" strategy instead of mapping each to
paddle.compat.X explicitly:

  1. api_mapping.json — change those 23 entries from
    ChangeAPIMatcher → paddle.compat.X to {"Matcher": "ChangePrefixMatcher"}, so
    torch.X(...) converts to paddle.X(...) with arguments untouched
    (e.g. torch.sort(a, dim=-1)paddle.sort(a, dim=-1)).
  2. transformer/import_transformer.py — inject import paddle +
    paddle.enable_compat() right after the imports of any converted file, so the
    prefix-only paddle.X calls resolve to the torch-aligned paddle.compat.X
    implementations at runtime. (Included explicitly + deduped so it is valid even
    when torch was imported as a submodule alias, e.g. import torch.nn as nn.)
  3. tests/apibase.pypaddle.enable_compat() flips global state (an
    import torch → Paddle proxy + paddle.* aliases). Disable it before each
    PyTorch reference run (_ensure_paddle_compat_disabled()), so the reference
    always executes against real PyTorch, including across multiple run() calls in
    one test. (No-op until Paddle is first imported, so it does not change torch/paddle
    import ordering.)

PR APIs

torch.allclose
torch.equal
torch.max
torch.median
torch.min
torch.nanmedian
torch.nn.AvgPool1d
torch.nn.AvgPool2d
torch.nn.AvgPool3d
torch.nn.Linear
torch.nn.MultiheadAttention
torch.nn.softmax
torch.nn.Unfold
torch.nn.functional.linear
torch.nn.functional.log_softmax
torch.nn.functional.pad
torch.nn.functional.scaled_dot_product_attention
torch.nn.functional.softmax
torch.nn.functional.unfold
torch.seed
torch.sort
torch.split
torch.unique

related Paddle PR:

Co-authored by Claude.

@Manfredss Manfredss closed this Jun 29, 2026
@Manfredss Manfredss reopened this Jun 29, 2026
@Manfredss
Manfredss marked this pull request as draft June 29, 2026 12:59
@Manfredss
Manfredss marked this pull request as ready for review June 29, 2026 13:00
@paddle-bot paddle-bot Bot added the contributor External developers label Jun 29, 2026
Manfredss and others added 6 commits July 2, 2026 20:37
… calls hit the alias

The default enable_compat() (level=1) does not alias paddle.*, so prefix-converted
calls (torch.X -> paddle.X) would run native and reject torch-style kwargs. level=2
turns on the paddle.* alias; caller-aware dispatch keeps paddle internals native.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…eset

import_transformer.py:
- Inject paddle.enable_compat(level=2) via a transform() post-pass so it lands
  after the docstring, __future__ imports and the import block (fixes a
  pre-existing SyntaxError where injected imports preceded `from __future__`).
- Gate on real torch imports (torch_packages), not paddle_package_list, so a
  torch-free file importing only os/einops/setuptools no longer gains a needless
  paddle import or the process-global compat switch.

tests:
- Rename confest.py -> conftest.py so the autouse compat-reset fixture is loaded
  by pytest (was dead due to the typo).
- Dedupe the disable logic into conftest.disable_paddle_compat(), imported by
  apibase and called before each torch reference exec.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Manfredss added 4 commits July 8, 2026 08:22
…ompat

Switch Tensor.allclose and Tensor.sort to ChangePrefixMatcher and keep
softmin's softmax call using torch arg names (input/dim), since compat
paddle APIs now accept torch-style args and reject paddle-native ones.
Update tests to inject enable_compat(level=2); fix torchvision
model_apibase proxy leak around the torch reference forward.
# enable_compat is injected in transform() (needs all imports in the body).
# Gate on real torch imports, not paddle_package_list: the latter also holds
# MAY_TORCH packages (os/einops/setuptools) that need no compat switch.
if self.imports_map[self.file]["torch_packages"]:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

简化下代码,看有无更简单的写法,是否只需要修改visit_Module就可以?

@zhwesky2010

Copy link
Copy Markdown
Collaborator

PR冲突了

@zhwesky2010 zhwesky2010 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这些插入操作能不能复用已有的函数record_scope

@zhwesky2010

zhwesky2010 commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

注意代码风格简化并贴合已有架构来写,不要让AI随意发挥:

  1. 贴合已有架构、功能和风格来开发,不要从0开始(AI不懂代码库架构 喜欢重起炉灶)
  2. 非代码复用的情况下,不需要封各种类函数(只会增加代码复杂度并无意义)
  3. 移除冗余注释

return True
return False

def _inject_enable_compat(self):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个使用record_scope不能插入吗?插入这个不需要从0开始重写吧

Manfredss and others added 2 commits July 14, 2026 02:53
Bring the latest API compatibility coverage into the branch while preserving the record_scope-based compat injection and test state isolation fixes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@paddle-bot

paddle-bot Bot commented Jul 14, 2026

Copy link
Copy Markdown

Thanks for your contribution!

Manfredss and others added 2 commits July 14, 2026 06:40
Align expected diffs with compat injection and intentional ChangePrefixMatcher behavior.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Deduplicate paddle_package_list in ImportTransformer.visit_Module: since
imports are now batched into one record_scope call, its cross-call dedup
no longer removes duplicates, producing two 'import paddle' lines when a
file imports multiple torch-family packages (e.g. torchvision + datasets).

Regenerate default-mode goldens for the paddle.enable_compat(level=2)
injection and the intentional compat-prefix removals (equal, BatchNorm1d/2d),
matching the min-mode baseline update in f7e1c04.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@zhwesky2010 zhwesky2010 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

简单问题不用复杂化:这里的node进来时已经是一个处理干净的node,所有torch相关的import都清除干净了,只用在这个基础上插入好paddle相关import即可

line_NO += 1
import_nodes = []
for paddle_package in dict.fromkeys(paddle_package_list):
import_nodes.extend(ast.parse(f"import {paddle_package}").body)

@zhwesky2010 zhwesky2010 Jul 14, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

直接在这里判断是否为 paddle_package=='paddle'

如果是,则import_nodes.extend(ast.parse(f"import paddle;paddle.enable_compat(2)").body)

并且该操作仅执行一次(当然执行多次问题也不大,后续会被isort校正)


has_torch_package = bool(self.imports_map[self.file]["torch_packages"])

has_enable_compat = any(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这些判断都需要吗?后面也会经过isort、black的格式化处理,移除多余import,这里不要想的太复杂了

if has_torch_package and not has_enable_compat:
import_nodes.extend(ast.parse("paddle.enable_compat(level=2)").body)

import_end = 1 if node.body and ast.get_docstring(node, clean=False) else 0

@zhwesky2010 zhwesky2010 Jul 14, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for循环,self.record_scope((self.root, "body", line_NO), import_nodes) 就行了吧。

原始的node在visit_Import、visit_ImportFrom时就已经被清除,原始node的import内容应该已经被清理掉了。

另外加上log_info,这个是比较重要的调试信息

…a single paddle.enable_compat(level=2) on top of the already-cleaned node, dropping redundant enable_compat/torch-package scans and adding log_info
# Under `paddle.enable_compat(level=2)` the prefix-converted calls
# (torch.X -> paddle.X) resolve to the torch-aligned paddle.compat.* impls,
# so inject it once when `import paddle` is added. isort/black dedupe repeats.
if "paddle" in paddle_package_list:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个应该得 == "paddle" ,然后做个一次性判断 "enable_compat" not in import_code

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我看一下

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor External developers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants